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AN  OVERVIEW  OF  THE  AIR  TRAFFIC  CONTROL  SPECIALIST  AND 
FIRST-LINE  SUPERVISOR  SELECTION  SYSTEMS 

Jennifer  G.  Myers,  Ph.D. 


The  air  traffic  control  (ATC)  job  is  unique  in 
several  ways.  It  is  one  of  the  few  federal  jobs 
where  continued  employment  is  based  on  the 
successful  completion  of  a  selection  and  training 
program  wholely  conducted  by  the  government. 
The  skills  necessary  to  perform  the  job  are  not 
necessarily  those  acquired  through  formal  voca¬ 
tional  or  university  training,  although  university- 
based  ATC  training  has  recently  been  initiated  on 
a  test  basis.  Like  other  technical  occupations, 
reaching  journeyman  status  is  based  on  the 
successive  demonstration  of  the  acquisition  of 
increasingly  complex  job  skills. 

Individuals  interested  in  applying  for  an  ATC 
job  first  complete  a  battery  of  tests  administered 
by  the  Office  of  Personnel  Management  in 
different  regional  locations.  Eligibility  for  attend¬ 
ing  the  FAA  Academy  Screen  Program  in 
Oklahoma  City  is  based  on  a  composite  of  the 
test  battery  scores,  as  well  as  meeting  medical 
and  security  screening  criteria.  Following  the 
successful  completion  of  the  Screen  Program,  in¬ 
dividuals  are  assigned  to  a  field  facility  for  on- 
the-job  training.  The  length  of  field  training 
varies,  depending  on  the  type  of  facility  (en  route 
or  terminal),  and  ranges  from  about  1  to  3  years 
(see  Figure  1). 

Air  traffic  control  specialists  (ATCS)  who 
wish  to  advance  into  a  supervisory  position  must 
apply  to  the  Supervisory  Identification  and 
Development  Program  (SIDP).  This  is  an  agen¬ 
cy-developed  system  for  assessing  individual 
competencies  and  determining  eligibility  to 
compete  for  first-line  supervisory  positions.  The 
SIDP  was  first  proposed  in  response  to  the 
recommendations  of  congressional  committees 
following  the  air  traffic  controller  strike  in  1981. 
It  was  felt  that  by  basing  supervisory  selections 
on  factors  other  than  prior  training  and  technical 
performance,  the  organizational  culture  that 
precipitated  the  1981  strike  could  be  improved 
and  future  labor-management  problems  could  be 
avoided.  The  SIDP  was  implemented  on  a  test 
basis  in  2  regions,  Southwest  and  Northwest 


Mountain,  in  1985  and  continued  until  national 
implementation  of  the  program  in  1988.  Since 
1985,  over  10,000  supervisors  and  non-super¬ 
visors  have  applied  to  the  program  and  approxi¬ 
mately  5,000  of  those  applicants  have  successful¬ 
ly  completed  the  program. 

Continued  technical  competence  in  air  traffic 
control  is  required  in  the  supervisory  position 
because  task  performance  includes  supervision  of 
the  technical  work  of  others.  This  entails  a 
requirement  for  maintaining  operational  currency 
by  performing  in  an  operational  position  16 
hours  per  month.  However,  technical  competence 
is  but  one  of  the  competencies  assessed  in  the 
SIDP.  Rather,  the  SIDP  is  designed  to  allow 
applicants  to  demonstrate  other  skills  needed  for 
effective  supervision  of  people  and  programs, 
such  as  communications  and  interpersonal  skills. 
Hie  SIDP  is  a  multiple  hurdle  process  for  deter¬ 
mining  eligibility  for  supervisory  positions  (see 
Figure  2). 

Controllers  must  have  completed  one  year  at 
the  journeyman  level  and  received  a  fully  suc¬ 
cessful  rating  on  their  last  performance  appraisal 
to  be  eligible  to  apply  to  the  program.  The  Peer- 
Supervisory  Assessment  process  requires  the 
SIDP  applicant  to  identify  from  4  to  7  peers 
(depending  on  the  size  of  the  facility)  and  one  or 
more  supervisors  to  rate  him/her  on  4  perfor¬ 
mance  dimensions:  Interpersonal,  Communica¬ 
tions,  Direction  and  Motivation,  and  Technical 
Competence.  A  paired-comparison  method  is 
used  to  rate  each  applicant  against  other  ap¬ 
plicants  and  benchmarks  on  each  of  the  dimen¬ 
sions.  Supervisors  and  peer  ratings  are  combined 
to  produce  a  single  score  for  each  dimension. 
Dimension  ratings  are  averaged  to  create  a 
composite  score  and  the  top  applicants  (based  on 
their  composite  score)  are  referred  to  the  next 
stage  of  die  selection  process,  the  Skill-Based 
Interview  (SBI). 

The  SBI  is  a  combination  of  a  face-to-face 
interview  and  role  play  exercises.  In  the  face-to- 
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face  interview,  the  candidate  is  asked  8  questions 
to  assess  Organizational  Knowledge  and  Know¬ 
ledge  of  the  Supervisory  Role.  The  role-plays 
are  designed  to  simulate  realistic  situations  en¬ 
countered  as  a  supervisor.  Based  on  their  perfor¬ 
mance  in  the  scenarios,  applicants  are  rated  on  7 
competencies  (Problem  Solving  and  Analytical 
Ability,  Judgment,  Decisiveness,  Organizing  and 
Planning  Ability,  Interpersonal  skill,  Communi¬ 
cation  Skill,  and  Di  ection  and  Motivation)  by  a 
trained  panel  of  3  interviewers  and  receive  verbal 
and  written  feedback  following  the  interview.  A 
description  of  the  competencies  are  contained  in 
Appendix  A. 

There  are  3  possible  direct  outcomes  from 
the  SBI:  referral  to  (a)  self-development,  (b)  the 
Candidate  Review  Board,  or  (c)  the  Eligible  for 
Consideration  list.  Those  who  demonstrate 
ineffective  performance  in  the  SBI  are  counseled 
on  their  strengths  and  weaknesses  and  recommen¬ 
ded  for  self-development  prior  to  reapplying  to 
the  SIDP.  Those  who  perform  at  a  moderate 
level  are  referred  to  a  Candidate  Review  Board. 
The  board  determines  whether  the  individual  will 
be  placed  in  formal  development  for  skill  remed¬ 
iation  or  will  be  recommended  for  self-develop¬ 
ment  activities  before  reapplying  to  a  future 
announcement  of  the  SIDP.  Those  who  are 
assigned  to  formal  development  are  provided 
resources  for  receiving  training  and/or  additional 
experiences  (e.g.,  details  or  "shadowing"  assign¬ 
ments)  that  will  remediate  skill  deficiencies  and 
result  in  eligibility  for  supervisory  positions. 
Applicants  who  perform  at  a  high  level  on  each 
of  the  competencies  are  placed  on  an  Eligible  for 
Consideration  list  and  are  referred  to  position 
vacancies. 

Individuals  who  are  on  the  eligible  for  con¬ 
sideration  list  are  polled  for  their  interest  in 
applying  for  vacant  supervisory  positions.  Once 
a  list  of  eligible  and  interested  bidders  is  es¬ 
tablished,  the  list  is  sent  to  the  selecting  officials 
along  with  a  packet  of  information  containing  the 
PSA  and  SBI  results  and  work  history  informa¬ 
tion  (awards,  training,  and  job  experience)  for 
each  individual  referred  to  the  vacancy. 


advancement  and  recognition  in  nonsupervisory 
positions  is  based  on  the  achievement  of  a  high 
level  of  technical  competence.  However,  for 
promotion  to  a  first-line  supervisor  position, 
ATCSs  must  demonstrate  not  only  technical 
competence  but  also  skills  in  areas  such  as 
decision-making,  communication,  and  leadership. 
Several  questions  arise  in  the  selection  of  super¬ 
visors  from  a  technical  workforce:  Given  that 
technical  competence  is  required  to  perform  the 
first-line  supervisor  job,  does  prior  technical 
training  and  journeyman  performance  relate  to 
"promotability"  or  actual  selection  for  a  super¬ 
visor  position?  What  characteristics  distinguish 
those  who  are  selected  from  the  eligible  pool 
from  those  that  are  not  selected?  These  questions 
are  addressed  in  the  following  papers,  which 
examine  ATCS  entry  level,  screening,  and 
training  performance  in  relation  to  SIDP  perfor¬ 
mance. 


As  demonstrated  in  the  description  of  the 
ATCS  selection  and  training  program,  individual 
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APPENDIX  A 

FIRST-LINE  SUPERVISOR  COMPETENCIES 


ORGANIZATIONAL  KNOWLEDGE:  Demon¬ 
strates  knowledges  of  the  FAA  organizational 
components,  the  mission(s)  of  each  relevant 
organizational  unit,  and  the  principal  programs  in 
the  FAA. 

KNOWLEDGE  OF  SUPERVISORY  ROLE 
PERFORMANCE:  Displays  knowledge  of  the 
roles,  responsibilities,  and  duties  of  supervisors 
and  managers;  accurately  assesses  the  impact 
upon  others  of  role  performance,  and  supports 
and  promotes  organizational  decisions,  policies, 
programs  and  initiatives  such  as  EEO,  Employee 
Assistance  Program,  Survey-Feedback-Action 
Program,  and  Affirmative  Action. 

COMMUNICATION  SKILL:  Presents  and 
expresses  ideas  and  information  effectively  and 
concisely  in  an  oral  and/or  written  mode;  listens 
and  comprehends  what  others  are  saying;  shares 
information  with  others  and  facilitates  the  open 
exchange  of  ideas  and  information;  is  open, 
honest,  and  straightforward  with  others;  provides 
a  complete  and  timely  explanation  of  issues  and 
decisions  in  a  manner  appropriate  for  the  aud¬ 
ience;  and  present  information  and  material  in  a 
manner  which  gains  the  agreement  of  others. 

DECISIVENESS:  Makes  decisions,  renders 
judgments,  and  takes  action  on  difficult  or  un¬ 
pleasant  tasks  in  a  timely  fashion,  to  include  the 
appropriate  communication  of  both  negative  and 
positive  information  and  decisions. 

DIRECTION  AND  MOTIVATION:  Motivates 
and  provides  direction  in  the  activities  of  others 
to  accomplish  goals;  gains  the  respect  and  con¬ 
fidence  of  others;  appropriately  assigns  work  and 
authority  to  others  in  the  accomplishment  of 
goals;  provides  advice  and  assistance  as  required; 
and  establishes  high  quality  work  standards  for 
self  and  others. 

TECHNICAL  COMPETENCE:  Understands 
and  appropriately  applies  procedures,  require¬ 
ments,  regulations,  and  policies;  maintains 
credibility  with  others  on  technical  matters;  and 
uses  equipment,  procedures,  or  systems  in  the 


operational  and/oi  staff  environment  as  the 
position  requires. 

INTERPERSONAL  SKILL:  Is  aware  of,  re¬ 
sponds  to,  and  considers  the  needs,  feelings,  and 
capabilities  of  others;  deals  effectively  with 
others  in  both  favorable  and  unfavorable  situa¬ 
tions  regardless  of  their  status  or  position;  ac¬ 
cepts  interpersonal  and  cultural  differences; 
manages  conflicts,  confrontations,  and  disagree¬ 
ments  in  a  positive  manner  which  minimizes 
personal  impact,  to  include  controlling  one’s  own 
feelings  and  reactions;  and  provides  appropriate 
support  to  others. 

JUDGMENT:  Develops  and  evaluates  alternative 
courses  of  action;  makes  decisions  based  on 
correct  assumptions  concerning  resources  and 
guidelines;  supports  decisions  or  recommenda¬ 
tions  with  data  or  reasoning;  defines  and  imple¬ 
ments  solutions  to  problems;  and  recognizes 
when  no  action  is  required. 

PLANNING  AND  ORGANIZING:  Identifies 
requirements,  allocates,  and  effectively  uses 
information,  personnel,  time,  and  other  resources 
necessary  for  mission  accomplishment;  es¬ 
tablishes  appropriate  courses  of  action  for  self 
and/or  others  to  accomplish  specific  goals; 
develops  evaluation  criteria  and  tracking  systems 
for  monitoring  goal  progress  and  accomplish¬ 
ment;  and  specifies  objectives,  schedules,  and 
priorities. 

PROBLEM  SOLVING  AND  ANALYTICAL 
ABILITY:  Identifies  existing  and  potential 
problems;  notes,  understands  and  includes  the 
critical  elements  of  problem  situations;  obtains 
and  evaluates  relevant  information;  demonstrates 
awareness  that  new  and/or  additional  information 
sources  are  required;  notes  interrelationships 
among  elements;  identifies  possible  causes  of  the 
problems;  recognizes  the  need  to  shift  to  an 
alternative  course  of  action  including  innovative 
or  creative  approaches;  and  appropriately  ter¬ 
minates  information  collection  and  evaluation 
activities. 
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COGNITIVE  INDICATORS  OF  ATCS  TECHNICAL  ABILITY  AND 
PERFORMANCE  IN  A  SUPERVISORY  SELECTION  PROGRAM 

David  J.  Schroeder,  Ph.D. 


Historically,  first  level  supervisors  have  been 
selected  with  respect  to  their  technical  com¬ 
petence  and  their  performance  on  the  job  they 
will  supervise.  Raza  (1987),  in  a  comprehensive 
review  of  the  literature  concerning  the  selection 
of  first  level  supervisors,  concludes  that  while 
supervisors,  researchers,  and  human  resource 
personnel  have  for  a  number  of  years  felt  that 
interpersonal  skills  are  critically  important  factors 
in  the  performance  of  first-level  supervisors, 
current  selection  procedures  for  those  positions 
have,  by  and  large,  continued  to  emphasize 
technical  competence.  Furthermore,  Raza  (1987) 
provides  convincing  evidence  that  current  pro¬ 
cedures  are  less  than  optimal.  The  Supervisory 
Identification  and  Development  Program  (SIDP), 
developed  by  the  FAA,  is  designed  to  incor¬ 
porate  information  other  than  technical  perfor¬ 
mance  of  the  applicant  in  the  selection  process 
for  supervisory  Air  Traffic  Control  Specialists 
(ATCSs). 

Applicants  to  the  ATCS  job  must  first  com¬ 
plete  a  2-stage  selection  process;  a  battery  of 
selection  tests  administered  by  the  Office  of 
Personnel  Management  (OPM)  and  a  9  week, 
performance-based  Screen  program  at  the  FAA 
Academy  in  Oklahoma  City.  This  investigation 
was  developed  to  determine  the  relationship 
between  entry  level  information  on  applicants’ 
cognitive  aptitudes  for  performing  ATCS  work, 
as  measured  by  tests  in  the  OPM  selection  bat¬ 
tery,  and  subsequent  selection  as  a  potential 
supervisor  through  the  SIDP.  Specifically,  the 
interest  was  in  determining  if  scores  on  the  initial 
selection  tests  predict  technical  performance 
ratings  received  by  the  SIDP  applicants.  This 
analysis  was  undertaken  in  light  of  the  previous 
research  (Raza,  1987)  that  suggested  that  existing 
procedures  for  supervisory  positions  still  tend  to 
emphasize  technical  competence.  If  that  remains 
true  for  the  SIDP  process,  entry  level  measures 
of  technical  aptitude  may  well  predict  subsequent 
SIDP  performance. 


METHOD 

Subjects 

The  OPM  selection  battery  is  administered  to 
all  applicants  to  determine  hiring  eligibility. 
OPM  scores  were  entered  into  the  analytic  data 
base  for  2,493  ATCSs  who  applied  to  SIDP 
between  1985  and  1989.  Of  this  group,  data  were 
available  for  1,215  who  entered  into  the  en  route 
option  and  1,084  in  the  terminal  option.  This 
division  was  based  on  the  nature  of  the  Academy 
Screen  program  (terminal  or  en  route)  and  the 
type  of  facility  to  which  the  ATCSs  were  subse¬ 
quently  assigned  following  completion  of  the 
Academy  Screen.  The  2  types  of  facilities  do 
involve  significant  differences  in  the  overall 
complexity  and  type  of  air  traffic  they  control.  A 
small  sample  of  en  route  and  terminal  Academy 
entrants  also  completed  2  experimental  tests  -  the 
Directional  Headings  Test  DHT:  en  route 
N=238,  terminal  N=136  and  Dial  Reading  Test 
(DRT:  en  route  N=139,  terminal  N=134). 

Measures 

OPM  Selection  Battery.  The  current  OPM 
selection  battery  was  implemented  in  1981  (see 
Sells,  Dailey  and  Pickrel,  1984  for  information 
concerning  die  validation  of  the  test  battery)  and 
is  comprised  of  the  Multiplex  Controller  Aptitude 
Test  (MCAT),  the  Abstract  Reasoning  Test 
(ABSR),  and  the  Occupational  Knowledge  Test 
(OKT).  The  MCAT  is  a  speeded  paper  and 
pencil  test  that  presents  a  simulated  controller 
work  sample.  A  map  containing  several  aircraft 
on  various  air  routes  is  presented,  along  with 
tabular  data  concerning  the  aircrafts’  altitude, 
speed,  and  route  for  each  question.  Aircraft 
positions  and  interpretation  of  the  tabular  data  are 
required  to  resolve  questions  concerning  time  and 
distance  and  possible  conflicts  between  aircraft 
(see  Figure  1).  Harris  (1986),  in  a  study  relating 
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MCAT  scores  to  a  battery  of  cognitive  tests, 
concluded  that  the  MCAT  measures  "both  per¬ 
ceptual  and  cognitive  reasoning  abilities."  This 
result  was  consistent  with  the  research  findings  as 
reported  in  Sells,  Dailey,  and  Pickrel  (1984). 

The  ABSR  consists  of  a  series  of  figures  or 
letters  that  illustrate  principles  of  logic.  The 
objective  is  to  select  from  several  alternatives  the 
one  that  correctly  completes  a  sequence.  The 
ABSR  measures  aspects  of  logical  reasoning  and 
spatial-perceptual  ability  (see  Figure  2).  The 
OKT,  a  test  of  job  knowledge,  consists  of  ques¬ 
tions  about  air  traffic  control  phraseology  and 
procedures,  air  navigation,  and  aviation  weather. 
A  transformed  score,  referred  to  as  the  Trans¬ 
muted  Composite  (TMC),  is  derived  by  weight¬ 
ing  the  MCAT  and  ABSR  to  produce  a  distribu¬ 
tion  with  a  mean  of  70  and  a  maximum  score  of 
100.  The  final  OPM  rating  includes  the  addition 
of  extra  points  for  the  applicant’s  OKT  score  and 
veteran’s  preference  points.  OPM  selection 
battery  performance  measures  for  this  study 
included  the  raw  scores  for  the  MCAT,  ABSR, 
and  OKT  and  the  TMC. 

Dial  Reading  and  Directional  Heading  Tests. 
The  DRT  and  DHT  are  2  experimental  tests  that 
were  completed  by  a  small  sample  of  the  S1DP 
applicant  group.  The  DRT  is  a  brief  timed  test  in 
which  individuals  are  asked  to  identify  and 
correctly  interpret  seven  different  instrument 
dials  for  a  series  of  questions  (see  Figure  3).  The 
DRT  was  developed  in  the  1950’s  for  the  US  Air 
Force  to  select  candidates  for  undergraduate  pilot 
training.  Research  by  Boone  (1979),  Marshall- 
Mies  and  Colmen  (1976)  and  Schroeder,  Dollar 
and  Nye  (1990)  has  demonstrated  the  utility  of 
the  DRT  in  predicting  success  of  ATCS  trainees. 
The  DHT  is  a  speeded  paper  and  pencil  test  of 
spatial  ability.  Each  item  is  comprised  of  3  bits 
of  information  that  reflect  the  cardinal  points  on 
a  compass  (e.g.,  the  letter  "W",  a  symbol  "  *- ", 
and  the  notation  "270",  each  denote  "West",  see 
Figure  4).  Other  letters,  symbols,  and  notations 
refer  to  the  other  cardinal  points  on  a  compass 
(North,  East  or  South).  In  the  first  of  2  parts  of 
the  DHT,  the  individual  is  asked  to  determine  the 
direction  indicated  (if  all  information  is  consis¬ 
tent),  or  indicate  that  the  information  is  inconsis¬ 
tent.  In  the  second  part  of  the  test,  the  direction 


can  be  determined  if  2  or  more  of  the  3  types  of 
information  presented  agree;  the  direction  cannot 
be  determined  when  all  3  types  of  information 
disagree.  Research  has  also  demonstrated  that  the 
DHT  can  be  used  successfully  to  select  personnel 
for  ATCS  training  (Cobb  &  Matthews,  1972; 
Boone,  1979;  Schroeder,  Dollar  &  Nye,  1990). 

SIDP  Performance.  The  SIDP  process  was 
described  in  the  introduction  by  Myers  (1992). 
PSA  ratings  used  in  this  investigation  include:  the 
composite  (COMP);  communication  (COMM); 
interpersonal  skills  (INTR);  leadership  (LEAD); 
and  technical  competence  (TECH).  Rating  results 
from  the  Skill-Based  Interview  were  summed  to 
create  an  overall  score  (SBI  TOTAL). 

Analyses.  Means  and  standard  deviations  for 
the  OPM  selection  battery  were  obtained  for  the 
initial  ATCS  applicant  group,  entrants  to  the 
FAA  ATCS  Screen  program,  those  who  success¬ 
fully  completed  the  ATCS  Screen  program,  SIDP 
applicants,  SIDP  candidates  placed  on  the  list  of 
eligibles,  and  eligibles  selected  to  fill  supervisory 
positions.  Separate  analyses  were  completed  for 
ATCSs  in  the  terminal  and  en  route  options. 

Correlations  among  OPM  selection  tests, 
DHT,  and  DRT  with  the  PSA  and  SBI  com¬ 
ponents  of  the  SIDP  were  computed.  The  ana¬ 
lyses  were  conducted  separately  for  the  2  op¬ 
tions. 

RESULTS 

Means  and  standard  deviations  of  scores  on 
each  component  of  the  OPM  test  battery  for  en 
route  and  terminal  option  ATCSs  are  presented 
in  Table  1.  The  effects  of  restriction  in  range 
that  occur  as  part  of  the  selection  process  are 
evident  in  the  higher  MCAT  scores  of  Academy 
entrants  who  subsequently  enter  the  en  route 
option  (En  Route  M=90.0,  Terminal  M=88.5 
versus  M=69.7  for  all  applicants)  and  the  cor¬ 
responding  smaller  standard  deviations  (SD  of 
7.1  in  en  route;  7.3  terminal  versus  16.1  for 
applicants).  These  differences  were  also  mirrored 
in  the  ABSR  scores  for  both  options.  The  lack  of 
evidence  of  any  restriction  in  range  for  the  OKT 
is  due,  in  part,  to  the  small  number  of  applicants 
who  actually  have  an  ATC  background.  With  the 
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exception  of  a  lower  average  MCAT  score  for 
those  selected  as  super,  isors  in  the  terminal 
option  (88.2  versus  ^o.2  for  SIDP  eligibles-but 
not  selected),  there  was  little  difference  in 
MCAT  scopes  for  the  different  SIDP  groups  or 
for  those  selected  as  supervisors.  Supervisors 
sf’:*?ted  in  the  terminal  option  had  slightly  lower 
average  ABSR  scores  than  eligibles.  However 
within  the  en  route  option,  those  selected  had 
slightly  higher  ABSR  scores  (41.7  versus  40.6) 

The  largest  difference  between  the  eligible 
and  selected  supervisor  groups  was  evident  in  the 
OKT  scores.  Of  those  initially  entering  the  en 
route  option,  the  average  scores  for  supervisors 
(M  =  33.3)  was  statistically  below  (F(  1,466)  = 
7.4,  pC.Ol)  that  of  the  ATCSs  who  completed 
Academy  training  and  entered  the  same  option 
but  were  not  yet  selected  (M  =  38.1).  In  con¬ 
trast,  within  those  entering  the  terminal  option, 
eligibles  who  were  selected  exhibited  higher 
average  OKT  scores  (M  =  43.6)  than  those  who 
graduated  from  the  Academy,  applied  to  become 
a  supervisor,  or  were  placed  on  the  list  of  eli¬ 
gibles  (M  =  39.3,  F  (1,459)  =  7.6,  p<.01). 
The  present  analysis  suggests  that  individuals 
who  enter  the  terminal  option  with  a  more  exten¬ 
sive  background  knowledge  of  ATC  rules  and 
procedures  are  more  likely  to  be  selected  as 
supervisors,  while  the  opposite  proved  to  be  true 
in  the  en  route  environment.  However,  there  are 
many  other  factors  that  may  have  entered  into  the 
selection  process  that  were  not  examined  in  this 
study. 

While  4  of  the  correlations  between  the  selec¬ 
tion  and  experimental  test  scores  and  the  various 
SIDP  measures  for  ATCSs  in  the  en  route  option 
were  statistically  significant,  they  were  all  quite 
low  (see  Table  2).  Of  the  possible  comparisons, 
significant  correlations  were  noted  between  the 
MCAT  and  TMC  with  the  TECH  peer-supervisor 
ratings  (.07  and  .06)  and  between  ABSR  and  SBI 
TOTAL  (.  10).  Of  the  experimental  tests,  only  the 
DRT  exhibited  a  significant  correlation  with  any 
of  the  SIDP  measures;  a  correlation  of  .16  with 
TECH.  Thus,  there  is  weak  evidence  that  some 
aspects  of  cognitive  ability  at  the  time  of  entry 
into  the  ATCS  occupation  are  predictive  of 
subsequent  technical  ratings  and  likelihood  of 
being  selected  as  a  supervisor. 


As  was  true  for  the  en  route  option,  correla¬ 
tions  between  aptitude  scores  and  SIDP  measures 
for  terminal  option  applicants  were  low  (see 
Table  3).  Ratings  of  communication  skills  were 
positively  correlated  with  ABSR  (r  =  .06, 
p<  .05)  and  DRT  (r  =.22,  p<.05)  but  nega¬ 
tively  correlated  with  HT  (r  =  -.17,  p<  .05).  A 
higher  rating  on  interpersonal  skills  was  as¬ 
sociated  with  a  lower  score  on  the  OKT  (r  = 
.06,  p<  .04)  but  higher  ratings  on  ABSR  (r  = 
.08,  p<  .05)  and  DRT  ( r  =  .18,  •><  .05).  Tech¬ 
nical  competence  was  significantly  related  to 
DRT  scores  only  (r  =  .22,  p<  .05). 

Results  of  these  analyses  reveal  only  limited 
evidence  of  any  relationship  between  any  of  the 
aptitude  measures  and  either  the  peer-super¬ 
visory  ratings  or  the  overall  SBI  rating.  How¬ 
ever,  of  all  the  possible  correlations,  the  MCAT 
and  DRT  were  most  closely  correlated  with  peer- 
supervisory  ratings  2  of  the  technical  (TECH) 
competence  of  the  SIDP  applicants  in  comparison 
to  the  other  ratings.  This  finding  was  consistent 
across  both  options.  The  Abstract  Reasoning  Test 
(ABSR),  which  measures  a  related  yet  different 
aspect  of  the  cognitive  ability  of  ATCSs,  tended 
to  be  more  closely  correlated  with  some  of  the 
other  ratings,  including  the  overall  SBI  rating 
for  en  route  ATCSs. 

DISCUSSION 

These  results  offer  weak  support  for  the  con¬ 
cept  that  measures  of  aptitudes  for  the  ATCS  job 
taken  prior  to  entry  into  ATCS  training  are 
predictive  of  on-the-job  ratings  of  technical 
proficiency.  These  findings  are  of  increased 
interest  when  one  considers  the  amount  of 
time,  training,  and  other  activities  that  have 
transpired  between  initial  selection  and  subse¬ 
quent  application  to  become  a  supervisor.  Addi¬ 
tionally,  as  trainees  complete  the  various  stages 
from  selection  through  attainment  of  the  full 
performance  level  status,  continued  loss  of 
unsuccessful  trainees  ensures  that  the  remaining 
workforce  is  relatively  homogenous  with  respect 
to  technical  competence.  Perceptions  of  on-the- 
job  performance  are  therefore  more  likely  due  to 
the  presence  of  untapped  aptitudes  or  differences 
associated  with  various  personality  dimensions 
(eg.,  self  confidence,  assertiveness).  In  any 
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vent,  one  should  be  aware  that  the  aptitude 
leasures  included  in  this  study  represent  only  a 
united  number  of  cognitive  abilities  that  are 
ritical  for  on-the-job  performance  of  ATCSs.  It 
ould  be  that  a  broader  range  of  aptitude  mea- 
ures  would  reveal  more  consistent  and  higher 
orrelations  with  the  various  ratings  and  mea- 
ures  comprising  the  SIDP. 

Given  the  current  measures  of  aptitudes  for 
tie  ATC  profession,  there  was  little  support  for 
ny  relationship  between  these  aptitudes  and 
ither  overall  ratings  during  the  Skill  Based 
nterview  or  selection  as  a  supervisor.  However, 
ackground  knowledge  of  ATC  and  pilot  pro- 
edures  may  enhance  the  chance  of  an  indi- 
idual’s  selection  as  a  supervisor  for  those 
ntering  the  terminal  option.  One  possible  ex- 
ilanation  is  that  within  the  terminal  option,  die 
tTCSs  need  to  be  more  familiar  with  aircraft 
haract eristics  and  associated  rules  and  pro- 
edures  due  to  the  nature  of  the  work  activities. 

Therefore,  those  who  enter  the  terminal 
iption  with  a  more  extensive  experience  in  the 
irea  and  general  background  knowledge  are  per- 
eived  to  be  more  suited  for  a  supervisory  pos¬ 
tion.  However,  given  the  limited  number  of 
'ariables  included  in  this  study  and  the  large 
lumber  of  additional  factors  that  could  play  an 
explanatory  role,  additional  information  and 
uialyses  are  necessary  to  fully  understand  this 
iifference. 
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TABLE  1 

Descriptive  statistics  for  the  OPM  selection  battery 


MCAT  ABSR  OKT 

M  SD  M  SD  M  SD  N  Group 


Applicants 


69.7 

16.1 

30.5 

9.5 

28.8 

11.3  127,807 

1981-1985' 

En  Route 

90.0 

7.1 

39.8 

5.9 

36.2 

12.9 

5,993 

Academy  entrants' 

91.6 

6.5 

40.5 

5.8 

37.6 

13.2 

3,337 

Passed  Academy 

91.4 

6.5 

40.5 

5.7 

37.6 

12.8 

1,215 

SIDP  applicants 

90.8 

6.5 

40.7 

5.5 

37.5 

12.8 

468 

All  SIDP  eligibles 

90.8 

6.6 

40.6 

5.6 

38.1 

12.9 

407 

SIDP  eligible  -  not 

91.1 

6.1 

41.7 

4.9 

33.3 

11.4 

61 

Selected 

Terminal  Option 

88.5 

7.3 

39.1 

6.3 

39.4 

13.4 

3,095 

Academy  entrants' 

89.4 

6.9 

39.6 

6.1 

41.2 

13.7 

2,186 

Passed  Academy 

89.5 

6.9 

39.4 

6.3 

40.7 

13.4 

1,084 

SIDP  applicants 

89.8 

6.7 

39.4 

6.2 

40.1 

13.1 

461 

SIDP  eligibles 

90.2 

6.5 

39.6 

6.1 

39.3 

12.9 

85 

SIDP  eligible  -  not 

88.2 

7.3 

38.7 

6.6 

43.6 

13.8 

85 

Selected 

yet  selected 


yet  selected 


NOTES :  'From  Manning,  Della  Rocco,  and  Bryant  (1989) 
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TABLE  2 

Correlations  between  the  OPM  selection  tests,  experimented  tests 
and  selected  SIDP  ratings  for  applicants  entering  the  en  route  option 


OPM  &  EXPERI¬ 
MENTAL  TESTS 

Peer-Supervisory  Assessment  Ratings 

COMP 

COMM 

INTR 

LEAD 

TECH 

SBI  TOTAL 

MCAT 

.05 

.03 

.03 

.04 

.07* 

-.05 

TMC 

.04 

.03 

.02 

.03 

.06* 

-.01 

ABSR 

-.01 

.01 

-.01 

-.01 

-.01 

.10** 

OKT 

-.00 

-.02 

-.03 

.00 

.04 

.00 

DHT 

.02 

.04 

-.00 

.02 

.04 

.12 

DRT 

.06 

.04 

.02 

.02 

.16* 

-.08 

*p  <.  .05  **p  <.  .01 


TABLE  3 

Correlations  between  the  OPM  selection  tests,  experimental  tests  and 
selected  SIDP  ratings  for  applicants  entering  the  terminal  option 


Peer-Supervisory  Assessment  Ratings 


OPM  &  EXPERI- 


MENTAL  TESTS 

COMP 

COMM 

INTR 

LEAD 

TECH 

SBI  TOTAL 

MCAT 

.03 

.03 

.00 

.03 

.05 

.04 

TMC 

.05 

.05 

.03 

.04 

.05 

.05 

ABSR 

.05 

.06* 

.08* 

.05 

.01 

.04 

OKT 

-.03 

-.02 

-.06* 

-.01 

-.01 

-.02 

DHT 

-.16 

-.17 

-.15 

-.13 

-.14 

.15 

DRT 

.21* 

.22** 

.18* 

.16 

.22** 

.11 

*P  <.  -05  **p  <.  01 
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MULTIPLEX  CONTROLLER  APTITUDE  TEST  (MCAT) 

AIRCRAFT  ALTITUDE  SPEED  ROUTE 
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FIGURE  4 


AIR  TRAFFIC  CONTROL  SPECIALIST  TECHNICAL  COMPETENCE 
IN  INITIAL  TRAINING  AND  SELECTION  AS  A  FIRST-LINE  SUPERVISOR 

Pamela  S.  Della  Rocco,  M.A.  and  Dana  Broach,  Ph.D. 


A  first-line  supervisor  holds  a  key  spot  in  an 
organization,  serving  as  the  main  point  of  contact 
between  the  rank-and-file  and  management  and  as 
the  person  responsible  for  the  day-to-day  or¬ 
ganizational  performance.  The  selection  of  first- 
line  supervisors  was  considered  essen-tially 
settled  as  a  result  of  many  studies  between  1947 
and  1965  (Dooher  &  Marting,  1957;  Raza, 
1987).  Corporations  generally  selected  first-line 
supervisors  on  the  basis  of  their  technical  skills, 
rather  than  managerial  skills  (Northrup,  Cowin, 
Vanden  Plas,  Fulmer,  Bolick,  Bellace,  &  Rosen- 
zweig,  1978;  Patton,  1974). 

However,  evidence  from  indirect  sources 
suggests  that  such  selection  procedures  for  first 
line  supervisors  may  be  suboptimal.  For  ex¬ 
ample,  Raza  (1987)  noted  that  surveys  of  first 
line  supervisors  suggested  a  need  for  training  in 
communication  and  management  skills.  Myers 
(1990)  reached  much  the  same  conclusions 
through  an  analysis  of  the  FAA  biennial  Job 
Satisfaction  Survey  and  Survey-Feedback-Action 
Program.  Phillips  (1985)  argued  that  while 
technical  skills  and  experience  may  be  important, 
they  are  not  necessarily  related  to  the  skills  and 
abilities  required  of  a  successful  supervisor. 
Given  these  two  disparate  lines  of  research,  we 
asked  what  relationship  prior  technical  perfor¬ 
mance  in  training,  for  example,  had  to  promot- 
ability  and  ultimate  selection  for  supervisory 
positions.  To  answer  that  question,  we  inves¬ 
tigated  the  relationship  of  performance  in  initial, 
technical  air  traffic  control  specialist  (ATCS) 
training  to  selection  as  a  first-line  air  traffic 
control  supervisor  through  the  Federal  Aviation 
Administration  Supervisory  Identification  and 
Development  Program  (SIDP). 

Air  Traffic  Control  Specialist  Initial 
Qualification  Training 

Since  1976,  candidates  selected  for  the 
occupation  of  Air  Traffic  Control  Specialist 
(ATCS)  with  the  FAA,  in  the  En  Route  and 
Terminal  options,  were  required  to  complete  a  2 


stage  selection  process.  The  first  stage  consisted 
of  a  battery  of  written  tests  and  was  administered 
by  the  U.S.  Office  of  Personnel  Management 
(OPM,  formerly  the  Civil  Service  Commission) 
prior  to  hiring  by  the  FAA.  The  second  stage  of 
the  ATCS  selection  process  was  a  performance- 
based  pass/fail  initial  training  course  administered 
at  the  FAA  Academy  after  FAA  hire.  This  study 
focuses  on  ATCS  performance  in  the  Academy 
initial  qualification  courses  as  a  potential  predic¬ 
tor  of  selection  as  a  first  line  supervisor. 

ATCSs  work  in  1  of  3  options:  En  Route, 
Terminal,  and  Flight  Service  Station.  En  Route 
and  Terminal  controllers  ensure  the  separation 
of  aircraft  by  using  information  about  the  speed, 
direction,  and  altitude  of  aircraft  to  formulate 
clearances  and  communicate  them  to  pilots. 
Clearances  are  sets  of  instructions  for  pilots, 
designed  to  ensure  the  safe,  orderly,  and  ex¬ 
peditious  flow  of  air  traffic.  En  Route  controllers 
ensure  the  separation  of  aircraft  traveling  be¬ 
tween  airports.  There  are  2  types  of  terminal 
controllers.  Terminal  radar  approach  and  depar¬ 
ture  controllers  use  radar  to  separate  aircraft 
converging  on  or  departing  from  an  airport. 
Tower  cab  controllers  control  traffic  landing  or 
taking  off  from  the  airport,  most  often  without 
the  direct  use  of  radar.  First-line  En  Route  and 
Terminal  supervisors  oversee  and  coordinate  the 
actions  of  controller  teams  responsible  for  spec¬ 
ific  sectors,  or  segments  of  contiguous  airspace. 
Flight  Service  Station  specialists,  on  the  other 
hand,  provide  other  services  to  pilots,  such  as 
weather  briefings,  filing  flight  plans,  and  helping 
to  locate  lost  aircraft.  Flight  Service  Station 
specialists  do  not  issue  control  instructions  to 
aircraft;  rather,  they  relay  En  Route  or  Terminal 
controller  clearances,  as  appropriate.  Given  the 
fundamental  difference  in  the  options,  we  focused 
on  En  Route  and  Terminal  ATCSs  and  their 
supervisors,  as  they  are  responsible  for  issuing 
clearances  to  aircraft,  and  excluded  Flight  Ser¬ 
vice  Station  ATCSs  from  this  analysis. 

The  FAA  Academy  initial  qualification 
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courses  for  the  En  Route  and  Terminal  options 
were  essentially  based  upon  a  miniaturized 
training  testing  personnel  selection  model  (Siegel, 
1978,  1983;  Siegel  &  Bergman,  1975)  in  which 
individuals  with  no  prior  knowledge  of  the 
occupation  could  be  assessed  for  their  potential  to 
succeed  in  air  traffic  control.  Individuals  who 
failed  were  separated  from  the  occupation. 
Separate  En  Route  and  Terminal  Academy 
programs  were  administered  until  October  1985. 
All  candidates  included  in  the  present  study  were 
graduates  of  these  dual  programs.  Each  of  the 
screening  courses,  the  En  Route  Initial  Qualifica¬ 
tion  Training  and  the  Terminal  Initial  Qualifica¬ 
tion  Training,  trained  and  assessed  students  on 
the  application  of  nonradar  procedures  in  each 
type  of  airspace,  respectively.  Although  there 
were  a  number  of  specific  differences  between 
the  2  programs,  both  generally  consisted  of  an 
academic  and  a  laboratory  phase.  Didactic  class¬ 
room  training  on  nonradar  air  traffic  control 
(ATC)  rules  and  principles  was  given  in  the 
academic  phases  of  the  optionspecific  courses. 
Academic  content  included  aircraft  separation 
rules,  cooperative  agreements,  phraseology,  and 
flight  progress  strip-marking.  Students  then 
applied  these  rules,  principles,  and  procedures  of 
ATC  in  a  series  of  thirty-minute  scripted  scen¬ 
arios  in  the  laboratory  phases  of  the  dual  courses 
of  instruction.  Students  were  evaluated  in  acade¬ 
mics  based  upon  multiple  choice  examinations 
and  a  map  test  of  the  synthetic  airspace.  In  the 
laboratory,  students  were  graded  for  their  perfor¬ 
mance  on  6  scripted  scenarios.  A  final  timed, 
paper-and -pencil  exam  assessed  application  of 
ATC  rules.  A  composite  of  these  scores  deter¬ 
mined  pass/fail  status.  The  composite  score  was 
heavily  weighted  on  laboratory  performance.  A 
successful  candidate  was  required  to  achieve  a 
final  composite  score  of  70  or  better. 

Performance  in  the  ATCS  screening  courses 
appears  to  predict  future  performance  in  field 
training  reasonably  well.  For  example.  Manning, 
Della  Rocco,  &  Bryant  (1989)  reported  a  raw 
correlation  of  -.24  (p  <.  .01,  N  =  2,992)  be¬ 
tween  the  En  Route  Initial  Qualification  Course 
final  composite  scores  for  1981-1985  Academy 
graduates  and  status  in  En  Route  field  training. 
Field  training  status  in  that  study  was  coded  1  = 
Full  performance  level,  2  =  In  training  in  or¬ 


iginal  option ,  3  =  Switched  options,  and  4  = 
Failed.  Correcting  for  range  restriction  due  to 
explicit  selection  on  the  predictor  resulted  in  a 
corrected  correlation  of  -.44  between  En  Route 
Initial  Qualification  Course  scores  and  field 
training  status.  Manning  et  al.,  also  examined  the 
relationship  of  the  Terminal  Initial  Qualification 
Course  composite  final  scores  to  Terminal  train¬ 
ing  performance.  They  divided  Terminal  con¬ 
trollers  into  2  groups  for  that  analysis:  those 
assigned  to  Terminal  facilities  without  radar 
(visual  flight  rules,  or  "VFR";  N  =  441);  and 
those  assigned  to  Terminal  facilities  with  radar 
("Radar*;  N  —  966).  While  the  raw  correlations 
between  Terminal  course  scores  and  status  in 
VFR  training  was  non-significant  (r  =  -.08,  p  > 
.01),  the  initial  qualification  course  score  was 
significantly  related  to  various  indices  of  time  in 
training  at  specific  positions  within  the  tower 
cab.  In  contrast,  status  in  the  Radar  Terminal 
training  program  was  predicted  reasonably  well 
by  the  Terminal  Initial  Qualification  Course  final 
composite  scores  (r  =  -2%,p  <_  .01,  N  =  962). 
Adjusting  for  restriction  in  range  due  to  explicit 
selection  on  the  predictor  improved  that  overall 
correlation  to  -.51  between  Terminal  course 
scores  and  Radar  Terminal  training  status.  Over¬ 
all,  the  FA  A  Academy  ATCS  option-specific 
screening  programs  appeared  to  predict  field 
training  status  reasonably  well.  In  other  words, 
developmental  controllers  who  performed  well 
in  the  FAA  Academy  programs  appeared  to  go 
on  to  perform  well  in  technical  field  training. 
While  supporting  data  are  not  easily  available,  it 
may  be  reasonably  supposed  that  persons  who 
did  well  in  the  ATCS  initial  qualification  courses 
and  field  training  were  also  likely  to  do  well 
upon  reaching  full  performance  level. 

This  sophisticated  and  rigorous  screening  and 
on  the  job  training  system  assured  the  tech-nical 
competence  of  the  ATCS  journeyman  or  full 
performance  level  (FPL)  workforce.  Systematic 
selection  of  first  line  supervisors  of  controllers  is 
accomplished  through  the  SIDP.  These  super¬ 
visors,  as  previously  noted,  oversee  controller 
teams  responsible  for  air  traffic  operating  within 
contiguous  areas  or  sectors  of  airspace  For 
example,  first-line  En  Route  ATCS  supervisors, 
or  Area  Supervisors,  carefully  monitor  current 
and  anticipated  activities  in  the  assigned  sectors 
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making  up  the  "area  of  specialization."  These 
Area  Supervisors  ensure  that  available  controller 
staffing  is  optimally  deployed  (Federal  Aviation 
Administration,  1989).  Technical  competence  is 
required  of  these  Area  Supervisors  to  interpret 
sector  and  area  activity,  and  to  render  assistance 
to  the  controller  teams  as  needed.  An  analogous 
situation  can  be  found  among  the  supervisors  on 
the  tower  lab  and  Terminal  Radar  Approach 
Control  (TRACON)  facilities.  These  supervisors 
are  selected  from  the  controller  workforce  thr¬ 
ough  the  SIDP. 

The  Supervisory  Identification 
and  Development  Program 

The  SIDP  was  designed  to  emphasize  fac¬ 
tors  such  as  communication  and  leadership,  in 
addition  to  technical  capability,  as  supervisory 
selection  criteria.  As  described  by  Myers  (1992), 
SIDP  utilized  a  2-stage  procedure  to  select 
supervisors.  Applicants  were  first  rated  by  peers 
and  supervisors  on  a  number  of  dimensions  in 
the  Peer/Supervisory  Assessment  (PSA)  stage  of 
SIDP.  Candidates  scoring  above  the  cutoff  on  a 
composite  of  PSA  ratings  then  participated  in  a 
Skill-Based  Interview  (SBI).  Candidates  who 
were  successful  in  the  SBI  were  then  placed  on 
an  "Eligible  for  Further  Consideration"  (EFC) 
list  from  which  supervisors  were  actually  selected 
as  openings  became  available. 

Because  it  may  be  important  that  supervi¬ 
sors  in  the  ATCS  occupation  demonstrate  tech¬ 
nical  competence,  we  hypothesized  that  success¬ 
ful  SIDP  candidates,  in  general,  would  demon¬ 
strate  better  technical  performance  as  early  in 
their  careers  as  the  FAA  Academy  initial  qualifi¬ 
cation  courses.  We  reasoned  that  (a)  if  persons 
doing  well  in  initial  training  did  well  in  field 
training,  as  the  available  data  suggested,  and  that 

(b)  if  persons  doing  well  as  FPL  controllers,  then 

(c)  such  persons  were  more  likely  to  be  selected 
into  first-line  supervisory  positions  through 
SIDP.  We  tested  this  reasoning  by  investigating 
the  following  3  research  questions: 

(1)  How  did  scores  in  the  initial  qualification 
training  courses  correlate  with  subsequent 
SIDP  PSA  measures  that  included  a  rating 
of  technical  competence? 


(2)  Did  SIDP  applicants,  selected  through  the 
SIDP  program,  demonstrate  better  technical 
skills  in  the  initial  qualification  courses  than 
unsuccessful  SIDP  applicants? 

(3)  Did  Academy  initial  qualification  course 
scores  predict  future  selection  as  a  first-level 
supervisor  in  the  SIDP  program? 

METHOD 

Subjects 

ATCSs  who  entered  the  FAA  Academy  in 
either  the  En  Route  or  Terminal  screening  pro¬ 
grams  between  August  1981  and  September  1985 
and  applied  to  the  SIDP  between  1985  and  1988 
were  selected  for  these  studies.  Some  applicants 
changed  options  after  they  passed  the  FAA 
Academy  initial  qualifications  programs.  For 
purposes  of  this  analysis,  only  those  applicants 
who  completed  the  Academy  and  were  still 
working  in  their  original  option  were  included. 
Of  the  1,128  applicants  meeting  these  sampling 
requirements,  468  were  in  the  En  Route  option, 
while  660  were  in  the  Terminal  option. 

All  of  the  controllers  in  this  sample  entered 
on  duty  with  the  FAA  after  the  illegal  1981 
strike  and  before  the  FAA  Academy  program 
changed  in  October,  1985.  The  sample  was 
predominantly  male  (85%)  and  nonminority 
(93%).  The  modal  formal  education  completed 
was  "some  college"  (48%),  with  about  a  third 
(33%)  having  completed  a  baccalaureate  degree. 
About  a  third  (34%)  had  at  least  some  prior 
aviation-related  experience  prior  to  joining  the 
agency. 

Three  groups  of  applicants  were  compared, 
based  upon  success  in  the  SIDP  selection  pro¬ 
cess:  (1)  candidates  who  were  successful  on  the 
PSA  and  referred  to  the  SBI  ( Referred  versus 
Not  referred );  (2)  based  upon  the  SBI,  candi¬ 
dates  were  deemed  "eligible  for  consideration" 
(EFC)  or  not  (£FC  versus  Not  EFC);  and  (3) 
candidates  selected  as  supervisors  from  the  EFC 
list  or  not  ( Selected  versus  Not  selected). 

Three-quarters  of  the  sample  (N  =  844)  were 
referred  to  the  Skill -Based  Interview  step  in  the 
SIDP.  About  half  (55%,  N  =  615)  of  the  total 
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sample,  or  three-quarters  ol  tnose  referred  to  the 
SBI,  were  rated  as  "Eligible  for  Further  Con¬ 
sideration"  in  the  SIDP.  Just  14%  ( N  =  160)  of 
the  total  sample,  or  26%  of  the  persons  rated  as 
"Eligible  for  Further  Consideration,"  were 
selected  as  first-line  supervisors  as  of  March 
1991.  Table  1  presents  data  on  applicant  status  in 
SIDP  by  career  option. 

Measures 

Thirteen  performance  measures  were  collec¬ 
ted  on  each  student  during  approximately  9 
weeks  at  the  FAA  Academy.  Five  of  the  meas¬ 
ures  were  multiple-choice  tests  covering  the 
academic  materials.  These  assessed  the  candi¬ 
date’s  ability  to  learn  and  retain  the  basic  know¬ 
ledge  required  for  foe  job.  A  sixth  measure  was 
a  map  test,  which  assessed  the  student’s  ability  to 
learn  a  map  of  foe  relatively  simple  synthetic 
airspace.  These  6  measures  accounted  for  10%  of 
foe  student’s  final  score.  For  purposes  of  this 
paper,  scores  from  4  multiple-choice  tests  and  foe 
map  test  were  combined  into  a  Block  Test  Aver¬ 
age  (BA).  The  fifth  multiple  choice  test  was  a 
comprehensive  final  exam  on  foe  academic 
phase,  foe  Comprehensive  Phase  Test  (CPT),  and 
was  analyzed  as  a  separate  measure. 

In  foe  laboratory  phase,  six  30-minute  stan¬ 
dardized  laboratory  problems  were  formally 
graded.  A  student  received  two  scores  from  a 
grading  instructor  on  each  problem.  The  first 
score,  foe  technical  assessment  (TA),  was  a 
numeric  assessment  of  foe  student’s  errors  in 
application  of  ATC  rules  and  procedures.  The 
second  score,  foe  instructor  assessment  (IA),  was 
a  subjective  instructor  rating  of  foe  student’s 
global  performance  on  foe  problem.  These  2 
scores  were  averaged  for  foe  final  problem  score. 
The  student’s  lowest  score  of  foe  6  laboratory 
problems  was  dropped.  Thus,  only  5  of  foe  6 
graded  problems  were  counted  toward  foe  final 
comprehensive  course  score.  The  graded  labor¬ 
atory  simulation  problems  comprised  65%  of  foe 
final  score.  Finally,  foe  students  were  given  a 
timed,  multiple-choice  test,  the  Controller  Skills 
Test  (CST),  which  was  a  paper-and-pencil  assess¬ 
ment  of  foe  student’s  ability  to  apply  ATC  rules 
and  procedures.  This  test  comprised  25%  of  foe 
Final  Comprehensive  Score  (COMP).  Table  2 


summarizes  these  FAA  Academy  initial  qualifi¬ 
cations  training  performance  measures. 

Analyses 

Because  foe  En  Route  and  Terminal  Initial 
Qualification  Courses  were  different  programs 
utilizing  measures  similar  in  name  only,  analyses 
were  conducted  for  foe  En  Route  and  terminal 
options  separately.  Analyses  were  conducted  to 
address  foe  following  research  questions: 

Research  Question  1 

What  was  the  relationship  between  FAA 
Academy  technical  performance  and  PSA  ratings, 
especially  foe  technical  competence  PSA  rating? 
We  hypothesized  that  there  would  be  a  non-zero 
correlation  between  the  FAA  Academy  measure 
of  technical  performance  and  PSA  ratings,  par¬ 
ticularly  the  PSA  technical  competence  category 
or  dimension. 

Research  Question  2 

Was  performance  in  FAA  Academy  initial 
technical  qualification  training  better  for  suc¬ 
cessful  than  for  unsuccessful  SIDP  applicants  at 
each  stage  of  SIDP?  We  hypothesized  that  there 
would  be  significant  differences  in  FAA  Acad¬ 
emy  scores  for  successful  compared  to  unsuc¬ 
cessful  participants  at  each  stage  of  SIDP.  We 
used  multiple  analysis  of  variance  (MANOVA)  to 
test  this  hypothesis. 

Research  Question  3 

Did  performance  in  foe  FAA  Academy  ini¬ 
tial  ATCS  qualification  training  program  predict 
success  at  each  stage  of  SIDP?  We  hypothesized 
that  higher  performance  in  the  FAA  Academy 
would  improve  foe  odds  for  success  at  each  stage 
of  SIDP.  We  used  logistic  regression  (Hosmer  & 
Lemeshow,  1989;  Norusis,  1990)  to  test  this 
hypothesis. 

RESULTS 

Research  question  1:  Academy  performance  - 
PSA  ratings 
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Because  the  PSA  contained  an  evaluation  of 
the  applicant’s  technical  abilities,  an  analysis  was 
conducted  to  determine  the  extent  of  the  relation¬ 
ship  between  Academy  measures  and  PSA  rat¬ 
ings.  Table  3  presents  the  correlations  among  the 
SIDP  PSA  measures  and  Academy  scores  for  the 
En  Route  ami  Terminal  options.  Results  of  these 
analyses  revealed  that  there  were  small  but 
statistically  significant  correlations  between  all  of 
the  Academy  measures  and  the  PSA  Technical 
rating,  except  for  the  CPT,  for  the  En  Route 
option.  In  the  Terminal  option,  however,  the 
hypothesized  correlations  between  the  PSA 
Technical  ratings  and  laboratory  measures  were 
not  found. 

Research  question  2:  Academy  performance  - 
SIDP  outcomes 

The  purpose  of  research  question  2  was  to 
compare  Academy  performance  of  successful 
SIDP  applicants  to  unsuccessful  applicants  at 
each  stage  of  the  SIDP  selection  process.  We 
hypothesized  that  successful  SIDP  applicants 
demonstrated  better  technical  performance  as 
early  in  their  careers  as  the  Academy  initial 
qualification  training  courses.  Table  4  presents 
descriptive  statistics  for  the  SIDP  applicants’ 
Academy  performance  scores  by  outcome  at  each 
stage  of  SIDP.  Results  of  MANOVAs  utilizing 
the  Academy  variables  as  dependent  variables 
revealed  no  statistically  significant  differences 
between  groups  for  (1)  referral  to  SBI  for  either 
En  Route  or  Terminal,  (2)  eligibility  for  con¬ 
sideration  for  Terminal  only,  and  (3)  selection 
for  either  En  Route  or  Terminal  supervisor.  A 
MANOVA  comparing  the  En  Route  EFC  and  Not 
EFC  groups  was  significant  (F  =  2.73,  p  < 
.01).  Univariate  analyses  of  variance  (df  =  1, 
349  in  each  analysis)  demonstrated  significant 
differences  between  EFC  and  Not  EFC  groups  on 
the  following  Academy  variables:  AVL5  (F  = 
7.33,  p  <  .01);  AVIA  (F  =  12.96,  p  <.  .001); 
A  VTA  (F  =  5.61,  p  <  .05);  and  COMP  (F  = 
4.83,  p  <_  .05).  The  mean  score  for  each  of 
these  variables  was  lower  for  the  EFC  group  than 
for  the  Not  EFC  group.  The  hypothesis  that 
successful  SIDP  candidates  had  demonstrated 
better  Academy  performance  than  their  unsuc¬ 
cessful  SIDP  counterparts  could  not  be  sup¬ 
ported. 


Research  question  3:  Academy  performance 
and  the  odds  for  success  in  SIDP 

As  ATCS  initial  qualification  course  scores 
predicted  technical  job  performance,  and  since 
that  technical  competence  was  required  of  con¬ 
troller  first-line  supervisors,  we  hypothesized  that 
better  performance  in  the  ATCS  Academy  initial 
qualification  courses  improved  the  odds  for 
referral  to  the  Skill-Based  Interview,  for  making 
the  Eligible  for  Consideration  list,  and  for  selec¬ 
tion  as  a  supervisor.  The  independent  variables, 
representing  Academy  performance,  were:  acade¬ 
mics  block  test  average  (BA);  academics  com¬ 
prehensive  phase  test  (CPT);  average  of  the  6 
graded  laboratory  problem  technical  performance 
assessments  (A VTA);  average  of  the  6  graded 
laboratory  problem  instructor  assessments  of 
potential  (AVIA);  and  score  on  the  final  Con¬ 
troller  Skills  Test  (CST).  These  independent 
variables  were  logistically  regressed  on  the 
categorical  outcomes  at  each  stage  of  SIDP. 
Results  of  the  logistic  regression,  used  to  test  this 
hypothesis,  are  presented  below. 

Referral  to  SBI.  The  probability  of  being 
referred  to  the  SBI  was  estimated  using  the  SPSS 
logistic  regression  procedure  (Norusis,  1990);  the 
independent  variables  were  entered  into  the 
equation,  and  model  goodness-of-fit  evaluated. 
The  analysis  indicated  that  previous  performance 
in  technical  training  did  not  predict  referral  to 
SBI  in  either  the  En  Route  or  Terminal  options. 
Assessment  of  the  goodness-of-fit  using  -2  times 
the  log  likelihood  (-2  LL)  of  the  model  indicated 
a  poor  fit  to  the  data  (En  Route:  -2  LL 
XJ(467)= 526.3,  p  <  .05;  Terminal:  -2  LL 
Xl(658)  =  737.0,  p  <  .01),  where  the  null 
hypothesis  was  that  the  model  fit  the  data. 
Inspection  of  the  classification  table  resulting 
from  the  model  supported  the  conclusion  of  a 
poor  model  fit.  While  the  overall  correct 
classification  rate  of  75.0%  and  74.7%,  for  En 
Route  and  Terminal,  respectively,  appeared 
acceptable,  all  but  2  of  the  Terminal  cases  not 
actually  referred  to  the  SBI  were  predicted  as 
referrals.  In  other  words,  the  model  predicted 
that  all,  except  2,  cases  would  have  been  referred 
to  the  SBI.  Thus,  performance  in  the  ATCS 
initial  qualification  courses  did  not  provide  useful 
information  about  characteristics  predicting 
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eferral  to  the  Skill-Based  Interview. 

Eligibility  for  Consideration.  The  probability 
)f  making  the  "Eligible  for  Consideration"  list 
ifter  referral  to  the  SBI  was  estimated  by 
:ntering  the  independent  variables  into  the 
equation,  and  evaluating  model  goodness-of-fit. 
rhe  analysis  indicated  that  previous  technical 
raining  performance  did  not  predict  making  the 
EFC  list  from  the  Skill-Based  Interview.  Assess¬ 
ment  of  the  goodness-of-fit  using  the  -2  times  the 
log  likelihood  of  the  model  indicated  a  poor  fit  to 
he  EFC  data  (En  Route:  -2  LL  x\348)  = 
199.2,  p  <  .05;  Terminal:  -2  LL  x*(491)  = 
$63.4,  p  <  .05)  under  the  null  hypothesis  that 
he  model  did  fit  the  data.  The  overall  rate  of 
:orrect  classifications  by  the  model  was  72.6% 
for  En  Route  and  73.4%  for  Terminal.  However, 
he  model  predicted  that  all  but  6  En  Route  and 

1  Terminal  applicant  would  have  made  the  EFC 
list.  This  result  indicated  that  performance  in 
initial  technical  training  failed  to  provide  useful 
information  for  the  practical  prediction  of  making 
the  EFC  list  from  the  interview  process. 

Selection  as  a  supervisor.  Finally,  the  prob¬ 
ability  of  selection  from  the  EFC  list  as  a  super¬ 
visor  was  estimated  by  once  again  entering  the 
independent  variables  into  the  equation  and 
assessing  model  goodness-of-fit.  This  analysis 
also  indicated  that  previous  performance  in  initial 
training  did  not  predict  actual  selection  for  a 
first-line  supervisory  position  in  the  air  traffic 
controller  occupation.  The  null  goodness-of-fit 
hypothesis  that  the  model  fit  the  data  for  the 
Terminal  option  was  rejected  based  on  -2  times 
the  log  likelihood  of  the  model  (-2  LL  xJ(361)  = 
428.4,  p  <  .01)  despite  an  overall  rate  of  70.8% 
correct  classifications.  This  time  the  model 
predicted  that  no  cases  would  been  selected  as  a 
supervisor  from  the  Terminal  option  EFC  list. 
For  the  En  Route  option,  the  null  goodness-of- 
fit  hypothesis  was  not  rejected  based  upon  the  - 

2  times  the  log  likelihood  of  the  model  (-2LL 
XJ(251)  =  761.9,  p  <  .31).  Although  the  overall 
correct  classifications  from  the  model  was 
78.6%,  no  selections  from  the  EFC  list  were 
predicted  based  upon  this  model  using  perfor¬ 
mance  in  initial  technical  training.  As  with  the 
previous  logistic  regression  analyses,  the  Acad¬ 
emy  measures  of  technical  performance  failed  to 


provide  useful  information  for  predicting  selec¬ 
tion  as  a  supervisor. 

DISCUSSION 

The  purpose  of  this  study  was  to  examine  the 
relationship  of  success  in  the  Supervisory  Iden¬ 
tification  and  Development  Program  and  prior 
performance  in  the  FAA  Academy  ATCS  initial 
qualification  courses.  Previous  studies  had  dem¬ 
onstrated  significant  correlations  between  Acad¬ 
emy  scores  and  subsequent  field  training  perfor¬ 
mance  (Manning,  Della  Rocco,  &  Bryant,  1989; 
Della  Rocco,  Manning,  &  Wing,  1990).  The 
basic  premise  that  persons  who  perform  well  in 
initial  technical  training  for  a  technically  demand¬ 
ing  occupation  would  be  more  likely  to  be  sel¬ 
ected  for  first-line  supervisory  positions  in  that 
occupation  was  tested  from  three  perspectives. 

The  first  research  question  examined  the 
relationship  between  Academy  scores  and  the 
ratings  from  the  Peer/Supervisory  Assessment. 
Small,  significant  correlations  were  found  be¬ 
tween  Academy  scores  and  the  PSA  technical 
competence  assessment  for  the  En  Route  option. 
Thus,  there  was  some  evidence  that  En  Route 
Academy  scores  were  related  to  Full  Perfor¬ 
mance  Level  (FPL)  or  journeyman-level  perfor¬ 
mance.  Hie  reason  that  the  correlations  were 
small  is  not  evident  because  greater  correlation 
has  been  reported  between  these  Academy  scores 
and  field  training  performance  (Manning,  et  al., 
1989).  However,  candidates  completed  the 
Academy  up  to  7  years  prior  to  the  assessment 
for  SIDP  which  is  a  considerable  length  of  time 
between  the  2  sets  of  assessments.  The  predicted 
relationship  between  Academy  laboratory  mea¬ 
sures  and  PSA  technical  ratings  was  not  found 
for  the  Terminal  option,  however.  This  finding 
corresponds  to  findings  of  Manning  et  al.,  (1989) 
of  the  correlations  between  the  Academy  and 
subsequent  field  training  measures  for  terminal 
option  controllers. 

The  second  research  question  assessed  the 
mean  differences  in  Academy  scores  between 
successful  and  unsuccessful  applicants  to  SIDP 
at  3  stages  of  the  SIDP  selection  process.  The 
hypothesis  that  successful  applicants  would  have 
performed  better  in  the  Academy  courses  was  not 
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supported.  Although  the  average  of  most  of  the 
Academy  scores  were  slightly  higher  for  the 
successful  candidates,  the  difference  was  not 
significant.  In  fact,  the  successful  candidates 
deemed  "eligible  for  further  consideration"  had 
statistically  significantly  lower  average  scores 
than  the  unsuccessful  candidates  in  the  En  Route 
option.  No  significant  differences  were  found 
between  any  of  the  SIDP  groups  in  the  Terminal 
option. 

The  final  research  question  investigated  the 
extent  to  which  Academy  scores  predicted  selec¬ 
tion  in  SIDP  using  logistic  regression.  The  an¬ 
alyses,  based  on  a  relatively  large  sample  of 
controllers,  indicated  that  performance  in  initial 
technical  training  was,  in  fact,  not  predictive  of 
referral  to  the  initial  stages  of  a  Supervisory 
Identification  and  Development  Program  nor  of 
selection  as  a  supervisor. 

On  one  hand,  these  data  lend  support  to  the 
contention  by  Phillips  (198S)  that  technical  skills 
and  experience  may  be  unrelated  to  the  skills  and 
abilities  required  of  supervisors.  On  the  other 
hand,  it  may  be  argued  that  the  initial  technical 
training  is  so  distant,  both  temporally  and  con¬ 
ceptually,  from  the  supervisory  selection  process 
as  to  be  meaningless.  Indicators  of  technical 
performance  measured  closer  in  time  to  con¬ 
sideration  for  SIDP,  such  as  performance  in  field 
training,  may  be  more  indicative  of  managerial 
potential.  Another  alternative  interpretation  is 
that  the  level  of  homogenous  technical  com¬ 
petence  within  the  workforce  is  assured  by  the 
nature  of  the  upfront  screening  process  repre¬ 
sented  by  the  ATCS  initial  qualification  courses. 
Thus,  other  factors,  such  as  communication 
skills,  knowledge  of  agency  programs,  may  be 
more  predictive  of  successfol  completion  of  the 
multiple  hurdles  embedded  within  SIDP. 
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TABLE  1 

Sample  status  at  each  stage  of  SID P  by  ATC  option 


Status 

Option 

En  Route 

Terminal 

Referral  to  Skill-Based  Interview  (SBI) 

Referred 

351 

494 

(15%) 

(75%) 

Not  Referred 

117 

166 

(25%) 

(25%) 

Eligible  for  Further  Consideration 

Eligible 

252 

363 

(72%) 

(73%) 

Not  Eligible 

99 

131 

(28%) 

(27%) 

Selected  for  a  Supervisory  Position 

Selected 

54 

106 

(21%) 

(29%) 

Not  Selected 

198 

257 

(79%) 

(71%) 
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TABLE  2 

FAA  Academy  ATCS  initial  qualifications  training  measures 


Phase 

Weight 

Measure 

Description 

Academics 

10% 

Block  test  average  (BA) 

Comprehensive  Phase  Test 
(CPT) 

4  multiple-choice  instructional  block  tests 
and  map  test 

comprehensive  multiple-choice  test  over  all 
academic  topics 

Laboratory 

65% 

Average  technical  score 
(A  VTA) 

Average  instructor  score 
(AVIA) 

Average  laboratory  score 
(AVL5) 

average  technical  performance  assessments 
cm  best  5  out  of  6  graded  laboratory 
problems 

average  instructor  assessment  of  potential 
on  best  5  out  of  6  graded  laboratory 
problems 

average  overall  score  on  best  5  out  of  6 
graded  laboratory  problems 

Final  examination 

25% 

Controller  Skills  Test  (CST) 

comprehensive,  multiple-choice  examination 

Final  score 

Final  composite  score  (COMP) 

70%  required  to  pass 

TABLE  3 

Correlations  between  FAA  Academy  performance  measures  and  PSA  ratings  by  ATC  option 


PSA  Rating  Categories 


Measure 

Communication 

Leadership 

Interpersonal 

Technical 

Composite 

En  Route 

BA 

.061 

.092* 

.080 

.104* 

.089 

CPT 

.059 

.085 

.050 

.081 

.073 

AVL5 

.034 

.023 

-.032 

.106* 

.032 

A  VTA 

.028 

.020 

-.034 

.097* 

.027 

AVIA 

.046 

.031 

-.013 

.107* 

.042 

CST 

.106* 

.118* 

.089 

.143* 

.120* 

COMP 

.068 

.065 

.006 

.144* 

.072 

Terminal 

BA 

.103* 

.081* 

.066 

.039 

.079* 

CPT 

.151* 

.128* 

.127* 

.084* 

.132* 

AVL5 

.040 

.044 

-.004 

.046 

.033 

A  VTA 

.056 

.070 

.016 

.063 

.055 

AVIA 

.007 

.027 

-.030 

.018 

.005 

CST 

.071 

.093* 

.043 

.081* 

.079* 

COMP 

.080* 

.090* 

.028 

.082* 

.075 

*  p<  .65 
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TABLE  4 

Comparison  of  FAA  Academy  performance  measures  by  ATC  option  and  outcome  at  each  stage  of  SEDP 
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RELATIONSHIPS  BETWEEN  PERFORMANCE  IN  AIR  TRAFFIC  CONTROL  SPECIALIST 
TECHNICAL  TRAINING  AND  SUPERVISORY  SELECTION  PROGRAMS 

Carol  A.  Manning,  Ph.D. 


After  successfully  completing  the  Federal 
Aviation  Administration’s  (FAA)  2-stage  selec¬ 
tion  procedure,  as  described  in  Della  Rocco  and 
Broach  (1991),  Air  Traffic  Control  Specialists 
(ATCSs)  enter  technical  training  programs  in 
their  respective  field  facility  assignments.  The 
type  and  duration  of  the  training  depends  on  the 
type  of  facility  (option)  and  amount  of  aircraft 
traffic  controlled  by  the  facility.  A  brief 
discussion  of  the  types  of  air  traffic  control 
facilities  will  be  presented,  followed  by  a 
description  of  the  training  programs  provided  by 
each  facility  type. 

ATCS  Options 

Air  Traffic  Controllers  in  the  FAA  can  be 
split  into  3  options  or  specialties:  en  route, 
terminal,  and  flight  service  station  (FSS).  En 
route  and  terminal  ATCSs  ensure  the  separation 
of  aircraft  traveling  between  airports  (en  route) 
and  approaching  or  departing  from  airports 
(terminal)  by  formulating  and  issuing  clearances 
(sets  of  instructions  for  aircraft  regarding  their 
appropriate  altitudes  and  directions  of  flight). 
The  clearances  are  designed  to  ensure  aircraft 
separation  and  maximize  fuel  efficiency.  Flight 
Service  Station  specialists  provide  services  to 
pilots  such  as  giving  weather  briefings,  filing 
flight  plans,  and  helping  to  locate  lost  aircraft. 
For  this  study,  the  relationship  between  technical 
training  performance  and  performance  in  the 
Supervisory  Identification  and  Development 
Program  (SIDP)  was  limited  to  en  route  and 
terminal  controllers,  because  the  screening, 
training  and  technical  requirements  of  the  FSS 
specialist’s  job  are  considerably  different  than 
those  for  ATCSs  in  the  other  options. 

ATCS  Field  Technical  Training 

In  spite  of  some  similarities  in  job  functions, 
the  technical  training  provided  for  en  route  and 
terminal  controllers  is  very  different,  and  even 
within  different  types  of  terminal  facilities,  the 
training  differs.  Much  of  the  reason  for  the  dif¬ 


ferences  in  training  is  that  air  traffic  control 
procedures  differ  according  to  the  facility  to 
which  the  student  is  assigned.  At  en  route 
facilities,  traffic  is  usually  moving  rapidly  and  at 
high  altitudes  and  must  be  kept  further  apart  than 
at  terminal  facilities,  where  traffic  is  slowing 
down  and  converging  on  a  single  location.  Thus, 
separation  standards  (the  minimum  distance  that 
aircraft  must  be  kept  apart)  vary  by  type  of  ATC 
facilities  and  the  training  varies  as  well. 

In  the  en  route  option,  the  unit  of  air  traffic 
control  operation  is  the  sector,  a  piece  of  air¬ 
space  for  which  a  team  of  2  or  3  controllers  is 
responsible  (during  times  of  slow  traffic,  only  1 
controller  may  be  responsible  for  a  sector).  A 
group  of  between  5  and  8  sectors  is  combined 
into  what  is  called  an  area  of  specialization.  An 
en  route  controller  is  assigned  to  only  1  area  of 
specialization,  but  is  responsible  for  learning  to 
control  traffic  in  all  sectors  within  that  area.  The 
team  of  en  route  controllers  working  at  most 
sectors  handles  duties  related  to  radar  separation 
of  aircraft  (radar  duties,  including  formulating 
clearances  to  ensure  separation  and  delivering 
them  by  radio  to  pilots,  handing  off  responsibility 
for  an  aircraft  to  another  controller),  duties  to 
assist  the  radar  controller  (radar  associate  duties, 
including  maintaining  records  about  clearances 
that  have  been  issued  or  other  changes  in  the 
flight  plan  of  an  aircraft,  identifying  potential 
problems,  communicating  information  not 
directly  related  to  aircraft  separation  of  aircraft  to 
pilots  or  other  controllers),  or  other  support 
activities  (assistant  controller  duties,  including 
entering  data  into  the  computer,  ensuring  that  all 
records  of  flight  progress  are  available  for  the 
controller  in  charge).  En  route  controllers  are 
trained  on  assistant  controller  duties  first,  then 
are  given  training  on  increasingly  difficult  re¬ 
sponsibilities  (radar  associate  duties,  then  radar 
duties).  Training  on  concepts  is  conducted  first 
in  the  classroom,  then  applied  in  a  laboratory 
setting,  then  reinforced  during  on-the-job  training 
(OJT)  conducted  in  a  supervised  setting.  At  some 
facilities,  ail  radar  associate  training  is  completed 
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on  each  sector  before  radar  training  begins;  at 
other  facilities,  both  radar  associate  and  radar 
training  are  provided  for  a  specific  sector  before 
training  begins  on  the  next  sector.  En  route 
controllers  must  complete  up  to  9  phases  of  field 
training,  depending  on  the  type  of  training 
provided  by  the  facility. 

In  the  terminal  option,  there  are  2  different 
types  of  controllers.  At  approach  control  facil¬ 
ities,  the  terminal  controllers  use  radar  equipment 
to  separate  aircraft  converging  on  or  departing 
from  an  airport,  although  these  aircraft  are 
slower  and  closer  together  than  they  are  in  the  en 
route  environment.  Approach  controllers  may 
work  in  pairs  but  often  work  alone  to  manage  the 
aircraft  within  their  radar  position  (equivalent  to 
an  en  route  sector).  Thus,  training  for  terminal 
approach  controllers  generally  involves  the 
combined  duties  of  a  radar  controller,  instead  of 
splitting  out  the  functions  into  several  training 
phases,  as  is  seen  in  the  en  route  environment. 
Also,  approach  controllers  can  be  required  to 
train  to  journeyman  level  in  up  to  10  sectors 
(called  "positions"  in  the  terminal  environment) 
in  comparison  to  a  maximum  of  8  sectors  for  en 
route  controllers.  Approach  controllers  take  at 
least  2  phases  of  field  training  (flight  data  and 
radar,  and  nonradar  training  is  optional)  but  may 
take  up  to  6  phases  if  their  facility  is  combined 
with  a  tower  cab. 

Tower  cab  controllers  control  traffic  landing 
or  taking  off  from  an  airport.  At  tower  cab 
facilities,  controllers  perform  duties  associated 
with  different  independent  functions,  e.g., 
(entering  and  updating  flight  data,  delivering 
clearances  related  to  the  flight  to  pilots  who  have 
not  yet  entered  the  taxiway,  directing  air  traffic 
moving  along  the  taxiways  back  and  forth  from 
the  runway,  and  directing  aircraft  to  take  off  and 
land).  Some  of  the  functions  may  be  combined, 
but  the  separate  functions  are  not  usually 
performed  as  a  team.  Thus,  training  is  conducted 
on  1  function  at  a  time.  Tower  cab  controllers 
complete  4  phases  of  field  training. 


Available  Measures  of  Field 
Training  Performance 

Several  types  of  information  on  performance 
are  obtained  for  each  phase  of  air  traffic  control 
field  training:  the  start  and  completion  dates,  the 
number  of  hours  used  to  complete  OJT,  and  the 
grade  (Pass,  Fail,  or  Withdraw).  A  rating  of 
controller  potential,  measured  on  a  1  to  6  scale, 
is  made  by  an  instructor  or  supervisor  who  most 
frequently  observed  die  student  during  that  phase. 
This  information  can  be  compiled  to  derive 
measures  of  training  performance,  such  as  the 
amount  of  time  (in  years)  required  to  reach 
journeyman  or  full  performance  level  (FPL) 
status,  mean  instructor  ratings  computed  across 
certain  training  phases,  time  (in  days)  to 
complete  OJT  in  certain  training  phases,  and  total 
number  of  OJT  hours  required  to  complete  those 
phases.  Total  OJT  hours  and  days  generally  vary 
by  facility  type  (Manning,  Della  Rocco  & 
Bryant,  1989).  For  example,  en  route  controllers, 
average  2.7  years  to  reach  FPL  status,  while 
terminal  approach  controllers  average  1.7  years; 
average  training  completion  time  for  tower  cab 
controllers  is  .7  years. 

Although  these  assessments  of  training  per¬ 
formance  are  available  for  most  students,  it  must 
be  understood  that  a  number  of  outside  factors 
(besides  aptitude  and  technical  proficiency)  may 
affect  the  accuracy  of  their  measurement.  For 
example,  time  to  reach  FPL  status  may  be 
affected  by  delays  in  training  caused  by  the  need 
to  use  the  controller  in  an  operational  position, 
the  number  of  other  students  undergoing  OJT  on 
the  same  airspace,  and/or  the  availability  of  the 
training  laboratory.  The  number  of  OJT  hours 
used  may  be  affected  by  insufficient  exposure  to 
different  types  of  traffic  during  training.  The 
subjective  rating  of  student  potential  could  be 
affected  by  a  number  of  rating  biases  familiar  to 
psychologists  (e.g.,  leniency,  central  tendency, 
severity,  halo  effect,  contrast  and  similarity 
errors).  Withdrawal  from  training  before 
completion  usually  occurs  because  of  training 
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failure,  but  students  occasionally  withdraw  from 
the  training  program  of  their  own  accord. 

Despite  the  measurement  problems  associated 
with  these  training  performance  measures,  they 
are  the  best  measures  currently  available  to 
describe  performance  in  ATCS  technical  training 
programs.  Some  of  the  measures  described 
above  have  been  used  as  criteria  against  which  to 
validate  ATCS  selection  procedures  (Manning, 
Kegg,  &  Bryant,  1989).  Statistically  significant 
(though  somewhat  small)  correlations  were  found 
between  the  Office  of  Personnel  Management 
rating  and  training  status,  instructor  ratings,  and 
time  to  reach  FPL  (a  negative  correlation)  for 
students  in  the  en  route  option;  somewhat  higher 
correlations  were  found  between  the  Academy 
score  and  the  same  measures  of  field  training 
performance. 

For  students  assigned  to  terminal  radar 
approach  control  facilities,  the  OPM  score  pre¬ 
dicted  training  status,  instructor  ratings,  and  time 
to  reach  FPL,  while  Academy  scores  only  pre¬ 
dicted  training  status  and  instructor  ratings.  For 
students  assigned  to  tower  cabs,  the  OPM  score 
was  not  predictive  of  any  measure  of  field  train¬ 
ing  performance.  However,  the  Occupational 
Knowledge  Test  score,  which  adds  extra  credit 
points  to  the  earned  rating  for  those  who  demon¬ 
strate  job  knowledge,  was  significantly  correlated 
with  days  and  hours  in  training  phases,  as  well 
as  time  to  reach  FPL  status  and  instructor  rat¬ 
ings.  The  Academy  score  was  significantly 
correlated  with  all  training  times  and  instructor 
ratings  for  students  at  tower  cab  facilities. 

For  this  study,  one  question  of  interest  was 
how  well  the  available  measures  of  field  training 
performance  would  predict  the  ratings  which 
determined  whether  the  candidate  progressed  to 
the  next  stage  of  supervisory  selection.  It  was 
expected  that  measures  reflecting  technical 
proficiency,  such  as  training  status  or  times  to 
complete  certain  portions  of  the  training 
program,  would  predict  ratings  of  technical 
competence  used  in  the  Peer-Supervisory 
Assessment  (PSA)  phase  of  the  SIDP  but  would 
be  unrelated  to  other  types  of  ratings.  On  the 
other  hand,  instructor  ratings  of  student  potential 
might  be  related  to  ratings  reflecting  other 


qualities  of  the  candidate,  such  as  interpersonal 
skills. 

Another  question  of  interest  was  how  well 
the  available  measures  of  field  training  perfor¬ 
mance  would  predict  success  in  the  3  stages  of 
the  supervisory  selection  process.  It  was  also 
expected  that  instructor  ratings  of  student 
potential  might  be  predictive  of  referral  to  the 
Skill-Based  Interview  (SBI),  which  is  based  on 
the  results  of  the  PSA.  On  the  other  hand,  it 
might  be  expected  that  none  of  the  measures 
would  igibility  for  Consideration  (EFC),  an 
assessment  made  by  an  independent  group  of 
raters. 

METHOD 

Subjects 

Records  of  field  training  performance  were 
obtained  for  1,352  applicants  to  the  SIDP  pro¬ 
gram  who  entered  the  FAA  Academy  between 
August  1981  and  September  1985,  successfully 
completed  that  program,  and  entered  ATCS  field 
training.  Three  records  were  discarded  because 
the  field  training  data  were  unavailable.  For 
other  records,  field  training  data  were 
occasionally  incomplete,  resulting  in  some 
missing  data. 

Training  Performance  Measures 

Because  of  the  differences  in  the  amount  and 
content  of  training  provided  to  ATCSs  assigned 
to  different  types  of  facilities,  some  measures  of 
field  training  performance  should  not  be 
compared  for  the  3  types  of  air  traffic 
controllers.  The  only  available  measures  of 
training  performance  appropriate  for  assessing 
general  technical  competence  are  instructor 
ratings  and  whether  or  not  the  controller  was 
successful  in  training  at  his  or  her  first  facility. 
(Controllers  who  are  unsuccessful  in  their  initial 
assignment  may  be  reassigned  to  a  facility  with 
lower  air  traffic  count  and  complexity.  Some 
controllers  who  failed  at  their  first  facility  could 
eventually  reach  FPL  status  at  a  subsequent 
facility  and  eventually  apply  for  the  SIDP 
program.)  However,  it  is  reasonable  to  analyze 
the  relationships  between  more  specific  measures 
of  field  training  performance,  such  as  the  amount 
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of  time  required  to  reach  FPL  status  and  times  to 
complete  specific  phases  of  training  for  students 
who  were  assigned  to  comparable  facilities.  Such 
analyses  were  conducted  for  controllers  assigned 
to  en  route,  terminal  approach  control,  and  tower 
cab  facilities. 

RESULTS 

Results  will  be  discussed  in  the  context  of 
research  questions  that  address  factors  which  are 
relevant  for  different  groupings  of  controllers. 

Question  1:  For  the  combined  group  of  all 
controllers,  how  does  proficiency  in  technical 
training  relate  to  success  in  SIDP? 

The  only  measures  of  technical  training 
performance  relevant  for  all  controllers  are 
instructor  ratings  and  overall  training  success. 
Table  1  shows  the  relationship  between  instructor 
ratings  and  status  in  the  different  stages  of  the 
SIDP  process.  Those  referred  to  the  SBI  had 
significantly  higher  instructor  ratings  than  those 
not  referred,  f(1217)=  2.37,/?  <  .02,  and  those 
selected  to  be  supervisors  had  significantly  higher 
instructor  ratings  than  those  not  selected,  r( 672) 
=  2.46,  p  <  .02.  However,  there  was  no 
difference  in  the  instructor  ratings  as  a  function 
of  SBI  eligibility  for  consideration  status. 

Table  2  shows  the  relationship  between 
training  success  and  SIDP  status.  The  likelihood 
of  being  successful  in  training  was  independent 
of  referral  to  the  SBI,  x*(l)  =  .0005,  p  >  .98, 
and  was  also  independent  of  eligibility  for  con¬ 
sideration,  xJ(l)  —  198,  p  >  .15.  However, 
training  success  was  related  to  supervisory 
selection,  *J(1)  =  5.54,  p  <  .02.  A  higher 
percentage  of  those  who  were  selected  for 
supervisory  positions  were  successful  in  training 
at  their  first  facility  than  those  who  were  not 
selected. 

Logistic  regression  analyses  were  conducted 
to  assess  the  relative  contribution  of  both 
instructor  rating  and  training  success  in 
predicting  status  in  the  3  stages  of  the  SIDP 
program.  Using  a  forward  selection  procedure, 
it  was  found  that  adding  the  mean  instructor 
rating  to  a  model  containing  no  variables  was  a 


significant  improvement  in  predicting  referral  for 
the  Skill  Based  Interview,  x*(l)  =  5.07,  p  < 
.03,  while  adding  training  status  to  that  model 
provided  no  significant  improvement,  x*(l)  = 
.18,/?  >  .65.  Neither  mean  instructor  rating  nor 
training  status  contributed  significantly  to  the 
prediction  of  eligibility  for  consideration.  Adding 
training  status  to  a  model  containing  no  variables 
significantly  improved  the  prediction  of 
supervisory  selection,  x2(l)  =  9.02,  p  <  .03, 
but  adding  mean  instructor  rating  to  that  model 
did  not  add  significantly  to  the  prediction,  x2(l) 
=  2.44,/?  >  .10. 

It  must  be  mentioned  that  the  goodness  of  fit 
of  the  model  is  somewhat  questionable;  the  -2 
Log  Likelihood  (-2  LL)  statistics,  indicating  how 
well  the  model  fits  the  data,  were  significantly 
different  from  1  for  the  prediction  of  SBI 
Referral  and  supervisory  selection;  -2  LL  (SBI 
referral)  =  1344.55,  p  <  .01,  df  =  1218,  and 
-2  LL(supervisory  selection)  =  752.49,/?  <  .02, 
df  =  673.  However,  the  goodness  of  fit  statistics 
for  the  same  models  did  not  lead  to  the  rejection 
of  the  models;  Goodness  of  fit  (SBI  referral)  = 
1219.0,  p  >  .48,  df  =  1218,  and  Goodness  of 
fit  (supervisory  selection)  =  674.0,  p  >  .48,  df 
=  673.  The  prediction  of  eligibility  for  con¬ 
sideration  by  the  logistic  regression  model 
resulted  in  a  classification  of  all  applicants  as 
"eligible,"  and  the  prediction  of  selection  by  the 
model  resulted  in  a  classification  of  all  candidates 
as  "not  selected."  Thus,  while  the  model 
successfully  predicted  EFC  and  supervisory 
selection  in  a  statistical  sense,  die  resulting 
classifications  predicted  by  the  model  are  of 
questionable  utility. 

Question  2:  For  en  route  controllers,  do  any 
of  the  specific  measures  of  training  perfor¬ 
mance  predict  success  in  SIDP? 

The  measures  of  technical  training  perfor¬ 
mance  relevant  to  the  analyses  concerning  the  en 
route  option  included  time  to  reach  FPL  status 
(in  years),  the  number  of  calendar  days  and  OJT 
hours  required  to  complete  training,  and  mean 
instructor  ratings  in  OJT.  Table  3  shows  means 
and  standard  deviations  for  the  field  training 
performance  measures  by  status  in  each  stage  of 
the  SIDP  program.  Those  referred  to  the  SBI  had 
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significantly  higher  instructor  ratings  than  those 
not  referred.  (F(l,456)  =  14.3,  p  C.001),  and 
used  significantly  fewer  OJT  hours  to  complete 
training  than  did  those  not  referred  (F( 1,453)  = 
4.49,  p  <  .04).  No  differences  in  field  training 
performance  as  a  function  of  eligibility  for 
consideration  were  observed.  Those  selected  as 
supervisors  spent  significantly  fewer  years  in 
training  (F( 1,239)  =  7.65,  p  <  .01),  ami 
significantly  fewer  days  in  OJT,  (F( 1,243)  = 
6.34,  p  <  .02),  than  did  those  not  selected. 

Logistic  regression  analyses  were  conducted 
to  assess  the  relative  contribution  of  the  training 
measures  in  predicting  status  in  each  phase  of 
SIDP.  A  model  was  developed  to  predict  SIDP 
status,  using  the  following  as  independent  vari¬ 
ables:  the  number  of  hours  and  calendar  days 
used  to  complete  OJT  on  the  first  2  sectors, 
mean  instructor  ratings  for  OJT  phases,  and  the 
amount  of  time  (in  years)  to  reach  FPL  status.  A 
model  containing  mean  instructor  rating  sig¬ 
nificantly  predicted  referral  to  the  SBI  (x*(l)  = 
16.5,  p  <  .001)  while  the  other  variables  con¬ 
tributed  nothing  more  to  the  prediction  of  the 
model  (x2(3)  =  1.95,  p  >  .58).  None  of  the 
variables  contributed  significantly  to  a  model 
predicting  eligibility  for  consideration  (x2(4)  = 
6.61,  p  >  .15).  A  model  containing  only  the 
time  to  reach  FPL  status  significantly  predicted 
supervisory  selection  (x2(l)  =  7.20,  p  <  .01), 
while  the  other  variables  contributed  nothing  to 
the  prediction  of  the  model  (x2(3)  =  4. 16,  p  > 
.24). 

It  must  be  noted  that  the  goodness  of  fit  of 
the  model  is  somewhat  questionable.  The  -2  LL 
statistic,  indicating  how  well  the  model  fits  the 
data,  was  significantly  different  from  1  for  the 
prediction  of  SBI  referral  status  (-2  LL  = 
488.15,  p  <  .04,  df  =  435)  but  the  goodness 
of  fit  statistics  for  the  same  model  did  not  lead 
to  the  rejection  of  the  model  (Goodness  of  fit  = 
436.0,  p  >  .47,  df  =  435.)  The  prediction  of 
eligibility  for  consideration  by  the  logistic 
regression  model  resulted  in  a  classification  of 
all  but  2  applicants  as  "eligible.”  Although  both 
goodness  of  fit  statistics  for  the  model  predicting 
supervisory  selection  suggested  that  the  model  fit 
the  data,  the  prediction  of  selection  by  the  model 
resulted  in  a  classification  of  all  candidates  as 


"not  selected."  Thus,  while  the  model  successful¬ 
ly  predicted  EFC  and  supervisory  selection  in  a 
statistical  sense,  the  classifications  predicted  by 
the  model  are  of  questionable  utility. 

Question  3:  For  en  route  controllers,  in  what 
ways  are  the  measures  of  training  per¬ 
formance  related  to  the  rating  dimensions  used 
to  determine  success  in  SIDP? 

Table  4  shows  descriptive  statistics  for  the 
predictors  and  rating  dimensions  for  en  route 
controllers,  and  Table  5  shows  correlations 
between  the  predictors  and  the  SIDP  rating 
dimensions.  Several  of  the  measures  of  training 
performance  had  statistically  significant,  though 
somewhat  low,  correlations  with  the  PSA  ratings, 
and  had  virtually  no  relationship  with  ratings 
from  the  SBI. 

Regression  analyses,  with  results  shown  in 
Table  6,  were  conducted  to  assess  the  relative 
contribution  of  the  training  measures  in  predict¬ 
ing  each  of  the  rating  dimensions  in  the  SIDP 
program.  While  the  training  performance  mea¬ 
sures  were  significantly  related  to  ratings  made 
on  PSA  dimensions,  very  little  relationship  was 
observed  between  training  measures  and  ratings 
made  on  the  SBI.  OJT  hours  and  the  number  of 
days  in  OJT  were  significantly  related  to  the 
composite  score,  and  the  Communications, 
Interpersonal  and  Leadership  rating  dimensions 
of  the  PSA.  A  model  containing  both  OJT  time 
and  instructor  rating  predicted  the  PSA  Technical 
rating  dimension.  However,  most  of  the  SBI 
rating  dimensions  were  not  predicted  by 
measures  of  training  performance.  The  number 
of  years  to  reach  FPL  status  had  a  modest 
relationship  with  ratings  on  Knowledge  of 
Supervisory  Role  Performance;  OJT  time  had  a 
modest  relationship  with  the  ratings  on 
Decisiveness.  Training  performance  measures 
accounted  for  10%  or  less  of  the  variance  in  the 
PSA  and  SBI  ratings. 

Question  4:  For  terminal  radar  approach 
controllers,  do  any  of  the  specific  measures  of 
training  performance  predict  success  in  SIDP? 

The  measures  of  technical  training  perfor¬ 
mance  relevant  to  the  analyses  dealing  with 
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terminal  radar  approach  controllers  included  time 
to  reach  FPL  status  (in  years),  calendar  days  and 
OJT  hours  required  to  complete  radar  training, 
and  an  instructor  rating  in  radar  training.  The 
training  days  and  OJT  hours  were  adjusted  by 
dividing  them  by  the  number  of  radar  positions 
in  the  airspace  for  which  their  facility  was  re¬ 
sponsible. 

Table  7  shows  means  and  standard  deviations 
for  the  field  training  performance  measures  by 
status  in  each  stage  of  the  SIDP  program.  Those 
referred  to  the  SBI  took  significantly  fewer  years 
to  reach  FPL  status  than  did  those  who  were  not 
referred,  F(l,314)  =  4.33,  p  <  .04.  Logistic 
regression  analyses  were  then  conducted  to  assess 
the  relative  contribution  of  the  training  measures 
in  predicting  status  in  each  phase  of  SIDP. 
A  model  was  developed  to  predict  SIDP  status, 
using  the  measures  described  above  as  indepen¬ 
dent  variables  and  the  3  stages  of  advancement 
in  SIDP  as  the  dependent  variables.  A  model 
containing  time  to  reach  FPL  status  marginally 
predicted  referral  to  the  Skill-Based  Interview, 
XJ(1)  =  3.74,  p  <  .06,  while  instructor  rating, 
OJT  hours,  and  number  of  days  spent  in  radar 
training  did  not  contribute  significantly  to  the 
model,  xJ(3)  =  4.09,  p  >  .25.  The  training 
variables  did  not  contribute  significantly  to  the 
prediction  of  eligibility  for  consideration,  nor  did 
they  contribute  to  the  prediction  of  supervisory 
selection. 

Although  both  goodness  of  fit  statistics  for 
the  model  predicting  referral  for  the  SBI 
suggested  that  the  model  fit  the  data  (-2  I L  = 
303.57,  p  >  .26,  df  =  289),  the  prediction  of 
eligibility  by  the  model  resulted  in  a  classification 
of  all  candidates  as  "eligible."  Thus,  while  the 
model  successfully  predicted  EFC  status  in  a 
statistical  sense,  the  classifications  are  not  useful. 

Question  5:  For  terminal  radar  approach 
controllers,  in  what  ways  are  the  measures  of 
training  performance  related  to  the  rating 
dimensions  used  to  determine  success  in  SIDP? 

Table  8  shows  descriptive  statistics  for  all 
predictors  and  criteria.  Table  9  shows  correla¬ 
tions  between  the  predictors  and  SIDP  rating 
dimensions.  The  number  of  years  to  attain  FPL 


status  was  significantly  correlated  with  several 
of  the  PSA  rating  dimensions,  but  only  with  the 
Supervisory  Role  rating  dimension  of  the  SBI  (in 
the  opposite  direction  than  was  predicted).  OJT 
hours  also  had  significant  negative  correlations 
with  the  PSA  rating  dimensions  but  were  posi¬ 
tively  correlated  with  the  Judgment  rating  dimen¬ 
sion  of  the  SBI. 

Regression  analyses,  shown  in  Table  10,  wore 
conducted  to  assess  the  relative  contribution  of 
the  training  measures  in  predicting  each  of  the 
rating  dimensions  in  the  SIDP  program.  Mea¬ 
sures  of  technical  training  performance  were 
again  more  highly  related  to  ratings  made  by 
peers  and  supervisors  than  ratings  made  by  those 
conducting  die  Skill-Based  Interview.  For  radar 
approach  controller  candidates,  OJT  hours  sig¬ 
nificantly  predicted  ratings  on  all  the  PSA  scales 
except  for  the  Technical  scale,  which  was  predic¬ 
ted  by  a  model  containing  both  instructor  ratings 
and  time  to  complete  training.  Regression  models 
containing  OJT  hours  also  predicted  ratings  on 
the  SBI  Judgment  and  Decisiveness  scales,  but 
the  relationship  was  not  in  the  predicted  direc¬ 
tion.  None  of  die  models  containing  measures  of 
training  performance  accounted  for  more  than 
10%  of  the  variance  in  the  ratings. 

Question  6:  For  tower  cab  controllers,  do  any 
of  the  specific  measures  of  training 
performance  predict  success  in  SIDP? 

The  measures  of  technical  training  perfor¬ 
mance  relevant  to  the  analyses  dealing  with  tower 
cab  controllers  included  time  to  reach  FPL  status 
(in  years),  calendar  days  and  OJT  hours  required 
to  complete  training  on  the  local  control  position, 
and  the  mean  instructor  rating  for  OJT.  Those 
referred  to  die  SBI  had  significandy  higher 
instructor  ratings  than  did  those  not  referred, 
FT1.230)  =  5.14,  p  <  .03.  No  significant 
differences  in  field  training  performance  were 
observed  as  a  function  of  eligibility  for 
consideration  or  supervisory  selection.  Logistic 
regression  analyses  were  conducted  to  assess  the 
relative  contribution  of  the  training  measures  in 
predicting  status  in  each  phase  of  SIDP.  A  model 
was  developed  to  predict  SIDP  status,  using  the 
measures  described  above  as  independent 
variables  and  status  in  the  3  stages  of  SIDP  as 
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the  dependent  variables.  None  of  the  measures  of 
field  training  performance  sufficiently  predicted 
any  stage  of  the  SIDP  process  to  justify  includ¬ 
ing  them  in  a  model. 

Question  7:  For  tower  cab  controllers,  in  what 
ways  are  the  measures  of  training  per¬ 
formance  related  to  the  rating  dimensions  used 
to  determine  success  in  SIDP? 

Table  12  shows  descriptive  statistics  for  all 
predictors  and  criteria.  Table  13  shows  correla¬ 
tions  of  the  predictors  with  PSA  and  SBI  rating 
dimensions.  The  only  significant  correlations 
were  between  the  training  measures  and  the 
Organizing/Planning  rating  dimension  of  the  SBI. 
Regression  analyses,  shown  in  Table  14,  were 
conducted  to  assess  the  relative  contribution  of 
the  training  measures  in  predicting  each  of  the 
rating  dimensions  in  the  SIDP  program.  For 
tower  cab  controllers,  none  of  the  training  per¬ 
formance  measures  significantly  predicted  the 
PSA  scales,  but  the  number  of  years  required  to 
reach  FPL  status  predicted  ratings  on  both  the 
Organizing/Planning  and  the  Direction/Motiva¬ 
tion  SBI  rating  dimensions.  However,  the  per¬ 
centage  of  variance  in  the  ratings  accounted  for 
by  the  number  of  years  to  reach  FPL  status  was 
less  than  5%. 

DISCUSSION 

The  purpose  of  this  study  was  to  assess  the 
contribution  of  measures  of  ATCS  field  training 
performance  in  predicting  1)  outcomes  in  the 
multiple  hurdles  of  the  Supervisory  Identification 
and  Development  Program,  2)  ratings  made  by 
peers  and  supervisors,  and  3)  ratings  of  know¬ 
ledges  and  skills  made  by  a  panel  during  role 
plays. 

Descriptive  statistics  were  computed  ami 
regression  and  logistic  regression  analyses  were 
conducted  for  all  terminal  and  en  route  con¬ 
trollers  combined.  The  Instructor  rating  in  on- 
the-job  training  was  related  to  referral  for  the 
SBI  and  training  status  was  related  to  supervisory 
selection. 

Analyses  were  also  conducted  separately  for 
en  route,  terminal  radar  approach  control,  and 


tower  cab  air  traffic  controllers  because  they 
perform  different  job  functions  and  undergo 
different  types  of  training.  The  measures  of  field 
training  performance  were  found  to  contribute  in 
different  ways  to  the  prediction  of  success  in 
SIDP  for  die  controllers  in  different  options.  In 
the  mi  route  option,  instructor  ratings  were 
related  to  referral  to  die  SBI,  and  die  amount  of 
time  required  to  reach  FPL  status  was  related  to 
supervisory  selection.  None  of  the  training 
measures  predicted  eligibility  for  consideration  as 
a  supervisor.  For  terminal  radar  approach 
controllers,  the  number  of  years  required  to 
reach  FPL  status  was  related  to  referral  to  die 
SBI,  but  none  of  the  training  measures  predicted 
eligibility  for  consideration  or  actual  selection. 
For  tower  cab  controllers,  none  of  the  training 
measures  contributed  significandy  to  the 
prediction  of  status  in  SIDP.  In  all  cases,  the 
logistic  regression  analyses  produced  models 
which  predicted  that  all  applicants  would  be 
referred  for  consideration  and  no  applicants 
would  be  selected.  Thus,  while  statistically 
significant  results  were  observed,  no  meaningful 
distinction  among  candidates  was  found. 

Additional  regression  analyses  were  conduc¬ 
ted  which  included  the  technical  training  perfor¬ 
mance  measures  that  were  specific  to  each  con¬ 
troller  option.  For  en  route  and  terminal  radar 
approach  controllers,  the  results  suggested  that 
candidates’  technical  proficiency  demonstrated  in 
training  was  related  to  subsequent  peer  and 
supervisor  ratings  of  technical  competence.  The 
measures  of  technical  proficiency  contributed  to 
the  prediction  of  ratings  made  by  peers  and 
supervisors  who  were  aware  of  the  candidates’ 
technical  skills.  On  the  other  hand,  those  who 
rated  candidates  during  the  SBI  did  not  rate  nor 
were  they  familiar  with  the  candidates’  technical 
proficiency.  Correspondingly,  the  measures  of 
candidates’  technical  training  performance  were 
generally  not  significantly  or  highly  predictive  of 
their  SBI  ratings.  This  outcome  is  as  expected; 
the  purpose  of  the  SBI  is  to  reinforce  selection 
on  factors  other  than  technical  competence  alone. 

Furthermore,  in  examining  more  closely  the 
correlations  between  measures  of  training  perfor- 
mance  and  PSA  ratings,  the  measures  of  the 
amount  of  time  required  to  complete  certain 
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training  phases  predicted  ratings  on  scales  pres¬ 
umably  unrelated  to  technical  skills-leadership, 
communication,  and  interpersonal  skills.  This 
finding  suggests  that  perceptions  of  technical 
proficiency  may  bias  how  peers  and  supervisors 
interpret  other  aspects  of  employees’  perfor¬ 
mance,  and  subsequently  influence  their  inter¬ 
pretation  of  supervisory  potential.  On  the  other 
hand,  the  rating  dimensions  may  tap  overlapping 
characteristics;  for  example,  peers  and 
supervisors  may  interpret  leadership  to  include 
technical  components,  thus  resulting  in  significant 
correlations  between  training  performance  and 
ratings  made  on  other  dimensions. 

Moreover,  measures  of  technical  proficiency 
and  instructor  ratings  both  contributed  to  the 
prediction  of  ratings  made  on  the  PSA  Technical 
scale,  although  the  instructor  rating  did  not 
contribute  significantly  to  regression  models 
predicting  ratings  made  on  any  of  the  other  PSA 
scales.  This  result  might  suggest  that  some  factor 
other  than  performance  alone  contributes  to  the 
assessment  of  technical  proficiency,  but  such  a 
speculation  is  not  confirmed  by  observing 
particularly  high  correlations  between  instructor 
ratings  and  other  PSA  rating  scales  that  might 
measure  this  other  factor  (such  as  communication 
or  interpersonal  skills). 

Finally,  similar  results  were  not  observed  for 
tower  cab  controllers.  One  reason  might  be  that 
insufficient  numbers  of  candidates  had  trained  at 
tower  cab  facilities  as  their  first  facility.  Also,  it 
is  possible  that  the  measures  of  training 
performance  do  not  describe  technical  proficiency 
for  the  tower  cab  controller  as  well  as  they  do 
for  other  types  of  controllers  who  spend  longer 
periods  of  time  in  training. 

In  summary,  available  measures  of  field 
training  performance,  though  hindered  by  mea¬ 
surement  problems,  are  somewhat  predictive  of 
both  ratings  of  future  potential  for  success  as  a 
first-line  supervisor  and  of  successfully 
completing  the  SIDP  program,  a  prerequisite  for 
becoming  a  first-line  supervisor.  In  one  sense, 
the  result  is  somewhat  predictable,  because  at 
least  minimal  technical  competence  is  a 
requirement  for  a  first-line  supervisor  (however, 
see  Slusher,  Van  Dyke,  &  Rose,  1972). 


Although  Della  Rocco  &  Broach,  (1991) 
suggested  that  technical  proficiency  measured  in 
the  FAA  Academy  was  not  predictive  of  SIDP 
outcomes,  the  training  performance  measures 
used  here  occurred  in  a  setting  that  resembled  the 
job  more  closely  than  did  the  AT C  screen 
program,  and  the  measures  were  obtained  more 
recently  than  were  the  screen  performance 
measures. 

On  the  other  hand,  the  results  are  somewhat 
unexpected,  because  a)  the  training  performance 
measures  are  fairly  global  and  do  not  describe 
specific  strengths  or  weaknesses  in  performance, 
b)  instructor  ratings  were  usually  made  by 
personnel  untrained  in  the  use  of  the  rating  scales 
(and  thus,  were  very  likely  tainted  by  a  number 
of  rater  biases),  c)  the  measures  were  obtained 
during  training  and  reflect  skill  learning  rather 
than  job  performance,  and  d)  a  number  of  other 
factors  besides  technical  success  are  assessed  in 
determining  success  in  the  SIDP  program.  At  the 
same  time,  it  must  be  remembered  that  none  of 
the  measures  of  training  performance  accounted 
for  more  than  10%  of  the  variance  in  ratings  of 
supervisory  potential  made  for  candidates  to  the 
SIDP  program.  At  best,  these  measures  might  be 
added  to  regression  models  containing  other 
variables  to  provide  additional  contribution  to  the 
prediction  of  SIDP  ratings. 

It  is  not  suggested  that  the  available  measures 
of  field  training  performance  sufficiently  describe 
controller  performance.  A  future  research  study 
is  planned  to  identify  and  develop  better  mea¬ 
sures  of  performance  during  training  and  on  the 
job.  When  those  measures  are  available,  it  will 
be  interesting  to  return  to  the  question  of  the 
relationship  between  technical  performance  and 
supervisory  success  and  determine  whether  the 
relationships  found  in  this  study  are  still  rele¬ 
vant. 
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TABLE  1 

Relationship  between  overall  instructor  rating  and  SIDP  status  for  all  ATC  options 


SIDP  Status 

M 

SD 

N 

Referred 

4.09 

.87 

926 

Not  referred 

3.96 

.85 

293 

Eligible 

4.09 

.86 

674 

Not  eligible 

4.11 

.88 

248 

Selected 

4.23 

.81 

166 

Not  selected 

4.04 

.88 

508 

TABLE  2 

Training  success  at  first  facility  and  SIDP  status  for  all  ATC  options 


^  — 

%  Not 

SIDP  Status 

Successful 

Successfid 

N 

Referred 

82.1 

17.9 

1018 

Not  referred 

82.2 

17.8 

331 

Eligible 

83.1 

16.9 

739 

Not  eligible 

79.3 

20.7 

275 

Selected 

88.8 

11.2 

179 

Not  selected 

81.3 

18.8 

560 
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Table  3 

Relationship  between  field  training  performance  and  status  in  SIDP  En  Route  controllers 
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Table  4 

Descriptive  statistics  for  predictors  and 
SIDP  rating  dimensions  for  En  Route  controllers 


Measures 

M 

SD 

N 

Training  Performance 

Instructor  Rating 

4.25 

.53 

458 

Years  to  reach  FPL 

2.60 

.59 

442 

OJT  Hours 

235.33 

88.68 

455 

QJT  Days 

142.17 

81.67 

454 

PSA  Ratings 

Composite 

61.04 

15.52 

467 

Communication 

62.01 

15.34 

466 

Interpersonal 

60.34 

17.57 

466 

Leadership 

60.22 

16.91 

466 

Technical 

61.59 

15.76 

466 

SBI  Ratings 

Agency  Programs 

3.66 

.83 

313 

Supervisory  Role 

3.84 

.81 

313 

Problem  Solv/Analytical 

3.96 

.89 

313 

Judgment 

3.82 

.94 

313 

Decisiveness 

4.07 

.87 

313 

Organizing/Planning 

3.93 

.91 

313 

Interpersonal  Skill 

4.21 

.85 

313 

Communication 

4.06 

.89 

313 

Direction/Motivation 

3.94 

.93 

313 
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Table  5 

Correlations  between  SIDP  rating  dimensions 
and  training  measures  for  En  Route  controllers 


Training  Measures 

SIDP  Ratings 

Years  to 

FPL  Status 

Instructor  OJT 

Rating  Hours 

OJT 

Days 

PSA 

Ratings  (N=435) 

Composite 

-.10* 

.24" 

-.20" 

-.18" 

Communication 

-.08 

.21" 

-.18" 

-.15" 

Interpersonal 

-.06 

.19" 

-.16" 

-.16“ 

Leadership 

-.11* 

.22" 

-.20" 

-.17" 

Technical 

-.14" 

.31" 

-.20" 

-.22" 

SBI 

Ratings  (N=291) 

Agency  Programs 

.01 

.02 

-.08 

.00 

Supervisory  Role 

-.13* 

.01 

-.01 

-.02 

Prob  Solv/Analytical 

.07 

-.05 

.00 

.00 

Judgment 

.08 

-.05 

-.07 

-.09 

Decisiveness 

.02 

.04 

-.06 

-.11 

Organizing/Planning 

.09 

-.06 

-.04 

-.05 

Interpersonal  Skill 

-.01 

-.05 

.02 

-.08 

Communication 

.03 

-.01 

-.03 

.08 

Direction/Motivation 

.05 

-.09 

.00 

.06 

*P  <  01  **p  <  .001 
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Table  6 

Results  of  regression  analyses  using 
field  training  performance  measures  as  predictors  of 
component  rating  dimension  scores  in  SIDP  En  Route  controllers 


Criterion 

Predictors 

R 

R 1 

0 

F 

P 

PSA  Ratings 

Composite 

OJT  Hours 

.23 

.05 

-.23 

16.13 

.0001 

Communication 

OJT  Hours 

.18 

.03 

-.18 

9.35 

.003 

Interpersonal 

OJT  Hours 

.18 

.03 

-.18 

9.57 

.003 

Leadership 

OJT  Hours 

.24 

.06 

-.24 

16.91 

.0001 

Technical 

OJT  Hours 

.32 

.10 

-.25 

16.60 

.0001 

Inst  Rating 

-.15 

SBI  Ratings 

Agency  Programs 

ns 

Supervisory  Role 

Years  to  FPL 

.14 

.02 

-.14 

5.35 

.03 

Problem  Solving 

ns 

Judgment 

ns 

Decisiveness 

OJT  Hours 

.12 

.01 

-.12 

4.14 

.05 

Organizing 

ns 

Interper  Skill 

ns 

Communication 

ns 

Direction/Motiv 

ns 
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Table  7 

Relationship  between  field  training  performance  and  status  in  SIDP  Terminal  Radar  Approach  controllers 
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Table  8 

Descriptive  statistics  for  predictors  and  SIDP 
rating  dimensions  for  Terminal  Radar  Approach  controllers 


Measures 

M 

SD 

N 

Training  Performance 

Instructor  Rating 

4.04 

.84 

368 

Years  to  reach  FPL 

1.84 

.87 

316 

OJT  Hours 

62.04 

40.60 

338 

OJT  Days 

95.06 

66.65 

339 

PSA  Ratings 

Composite 

57.66 

13.77 

378 

Communication 

59.49 

13.46 

377 

Interpersonal 

57.58 

15.32 

378 

Leadership 

56.38 

15.74 

378 

Technical 

57.38 

15.42 

378 

SBI  Ratings 

Agency  Programs 

3.84 

.79 

270 

Supervisory  Role 

3.88 

.76 

270 

Problem  Solv/ Analytical 

4.00 

.89 

270 

Judgment 

3.84 

.89 

270 

Decisiveness 

4.11 

.91 

270 

Organizing/Planning 

3.98 

.89 

270 

Interpersonal  Skill 

4.24 

.83 

270 

Communication 

4.13 

.80 

270 

Direction/Motivation 

3.96 

.91 

270 
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Table  9 

Correlations  between  SIDP  rating  dimensions 
and  training  measures  for  Terminal  Radar  Approach  controllers 


Training  Measures 

SIDP  Ratings 

Years  to 

FPL  Status 

Instructor  OJT 

Rating  Hours 

OJT 

Days 

PSA  Ratings  (N 

=  304) 

Composite 

-.12* 

.03 

-.15” 

-.10 

Communication 

-.07 

.00 

-.15” 

-.06 

Interpersonal 

-.03 

-.03 

-.14* 

-.06 

Leadership 

-.13* 

.02 

-.14* 

-.10 

Technical 

-.21” 

.09 

-.12* 

-.14* 

SBI  Ratings  ( N 

=  220) 

Agency  Programs 

.05 

-.00 

.02 

.00 

Supervisory  Role 

.13* 

.09 

-.06 

-.05 

Prob  Solv/Analytical 

.02 

-.04 

.11 

.07 

Judgment 

.05 

-.04 

.15* 

.05 

Decisiveness 

.03 

-.03 

.08 

-.01 

Organizing/Planning 

.06 

-.06 

.13 

.09 

Interpersonal  Skill 

.01 

-.07 

.12 

.05 

Communication 

.06 

-.10 

.08 

.02 

Direction/Motivation 

.01 

-.07 

.05 

-.01 

*p  <_  .05  **p  <.  .01 
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Table  10 

Results  of  regression  analyses  using  field  trainings 
performance  measures  as  predictors  of  component  rating 
dimension  scores  in  SIDP  Terminal  Radar  Approach  controllers 


Criterion 

Predictors 

R 

R 3 

0 

F 

P 

PSA 

Ratings 

Composite 

OJT  Hours 

.24 

.06 

-.24 

12.66 

.001 

Communication 

OJT  Hours 

.21 

.05 

-.21 

9.95 

.002 

Interpersonal 

OJT  Hours 

.23 

.05 

-.23 

11.38 

.001 

Leadership 

OJT  Hours 

.19 

.04 

-.19 

8.14 

.005 

Technical 

Inst  Rating 

.28 

.08 

.18 

8.68 

.001 

OJT  Days 

-.17 

SBI 

Ratings 

Agency  Programs 

ns 

Supervisory  Role 

ns 

Problem  Solving 

ns 

Judgment 

OJT  Hours 

.16 

.02 

.16 

5.30 

.03 

Decisiveness 

ns 

Organizing 

OJT  Hours 

.14 

.02 

.14 

3.97 

.05 

Interper  Skill 

ns 

Communication 

ns 

Direction/Motiv 

ns 
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Table  11 

Relationship  between  field  training  performance  and  status  in  SIDP  Tower  Cab  controllers 


Table  12 

Descriptive  statistics  for  predictors  and 
SIDP  rating  dimensions  for  Tower  Cab  controllers 


Measures 

M 

SD 

N 

Instructor  Rating 

Training  Performance 

4.23  .90 

232 

Years  to  reach  FPL 

.66 

.35 

240 

OJT  Hours 

104.17 

44.81 

238 

OJT  Days 

162.65 

103.38 

247 

Composite 

57.12 

PSA  Ratings 

15.52 

251 

Communication 

59.10 

15.88 

251 

Interpersonal 

56.84 

17.92 

251 

Leadership 

55.65 

17.18 

251 

Technical 

56.83 

16.62 

251 

Agency  Programs 

3.49 

SBI  Ratings 

.88 

162 

Supervisory  Role 

3.75 

.86 

162 

Problem  Solv/ Analytical 

3.85 

1.02 

162 

Judgment 

3.66 

.93 

162 

Decisiveness 

3.83 

1.04 

162 

Organizing/Planning 

3.75 

1.01 

162 

Interpersonal  Skill 

4.05 

.87 

162 

Communication 

3.88 

.99 

162 

Direction/Motivation 

3.76 

.98 

162 
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Table  13 

Correlations  oetween  SIDP  rating  dimensions 
and  training  measures  for  Tower  Cab  controllers 


Training  Measures 


SIDP  Ratings 


Years  to  Instructor  OJT  OJT 

FPL  Status  Rating  Hours  Days 


PSA  Ratings 


Composite 

Communication 

Interpersonal 

Leadership 

Technical 


Agency  Programs 
Supervisory  Role 
Prob  Solv/Analytical 
Judgment 
Decisiveness 
Organizing/Planning 
Interpersonal  Skill 
Communication 
Direction/Motivation 


.01 

.08 

-.01 

.06 

-.01 

.07 

.01 

.08 

.05 

.11 

SBI  Ratings 

.01 

.01 

-.09 

.13 

-.15 

.11 

-.08 

.07 

-.14 

.03 

-.20’ 

.13 

-.15 

.12 

-.15 

.11 

-.17’ 

.06 

-.00 

.01 

.01 

-.01 

.03 

.02 

-.00 

-.00 

-.05 

.03 

-.07 

-.04 

.03 

-.05 

-.09 

-.11 

-.04 

-.10 

-.09 

-.09 

.18* 

-.17’ 

-.08 

-.06 

-.04 

-.06 

-.12 

-.13 

*p  <_  .01  **p  <.  .001 
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Table  14 

Results  of  regression  analyses  using 
field  training  performance  measures  as  predictors  of 
component  rating  dimension  scores  in  SIDP  Tower  Cab  controllers 


Criterion 

Predictors 

R 

R 3 

0 

F 

P 

Agency  Programs 
Supervisory  Role 
Problem  Solving 
Judgment 
Decisiveness 
Organizing 

ns 

ns 

ns 

ns 

ns 

Years  to  FPL 

SBI  Ratings 

.20  .04 

-.20 

6.12 

.02 

Interper  Skill 

Communication 

Direction/Motiv 

ns 

ns 

Years  to  FPL 

.17 

.03 

-.17 

4.04 

.05 
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CANDIDATE  PERFORMANCE  IN  A  SUPERVISORY  SELECTION  PROGRAM 
AND  SUBSEQUENT  SELECTION  DECISIONS 

Jennifer  G.  Myers,  Ph.D. 


Small  surveys  of  private  organizations  have 
reported  that  fewer  than  half  of  these  organiza¬ 
tions  have  formal  selection  systems  in  piace  for 
first-line  supervisors  (Levine,  1986;  Rendero, 
1980).  Given  the  importance  of  first-line  super¬ 
visors  in  ensuring  employee  productivity  by 
directing  subordinate  work  activity,  as  well  as 
their  role  in  representing  management  and  the 
organizational  mission  to  subordinates,  die  lack 
of  reported  formal  supervisory  selection  systems 
is  somewhat  surprising.  Recent  research  on 
supervisory  selection  systems  and  promotion 
decisions  is  also  noticeably  absent. 

A  recent  review  of  federal  agencies’  super¬ 
visor  selection  programs  (U.  S.  Merit  Systems 
Protection  Board,  1989)  described  the  typical 
situation.  In  general,  applicants  to  first-line 
supervisory  jobs  document  training  and  exper¬ 
ience  information  typically  based  on  their  prior 
nonsupervisory  work  history.  As  noted  in  the 
review,  the  problem  for  the  selecting  official 
becomes  one  of  evaluating  an  individual's  ability 
or  potential  for  performing  supervisory  tasks  that 
have  never  been  done  before.  Given  the  makeup 
of  the  application  materials,  selecting  officials 
primarily  see  the  applicant’s  prior  technical  work 
experience.  Personal  interviews  with  applicants 
may  be  restricted  by  the  associated  costs,  since 
Merit  Promotion  Plan  regulations  require  all  or 
none  of  the  eligible  candidates  to  be  interviewed. 
The  frequent  result  in  the  federal  government  is 
that  the  most  highly  technically  qualified  person 
is  chosen  for  the  supervisory  job,  rather  than 
someone  with  demonstrated  management  skills. 
This  outcome  is  similar  to  that  found  in  private 
industry  (Northrop,  Cowin,  Vanden  Plas,  Ful¬ 
mer,  Bolick,  Bellace,  &  Rosenzwig,  1978). 
However,  selections  based  on  technical  perfor¬ 
mance  may  result  in  the  promotion  of  someone 
unsuitable  for  the  position,  since  technical  skills 
and  experience  are  not  necessarily  related  to 
supervisory  competence  (Phillips,  1985). 


Technical  competence  has  been  identified  as 
an  important  factor  in  the  performance  of 
supervisory  tasks  (Myers  &  Stutzman,  1991). 
However,  Myers  and  Stutzman  (1991)  found  that 
other  competencies  such  as  communications  skills 
and  interpersonal  skills  were  mentioned  more 
frequently  than  was  technical  competence  as 
important  in  successful  task  performance.  One 
method  of  ensuring  the  applicant  for  a  first-line 
supervisory  position  has  the  right  mix  of 
technical  and  supervisory  skills  is  to  formalize 
the  assessment  of  the  most  important 
competencies  through  a  structured  selection 
program.  Still,  selecting  officials  could 
emphasize  the  importance  of  technical  over 
supervisory  competence,  by  selectively  attending 
to  factors  in  the  work  history  that  demonstrate 
the  applicant’s  technical  competence.  As  already 
suggested  by  Phillips  (1985),  this  may  not  be  an 
optimal  strategy  for  selecting  the  best  candidate 
for  the  position.  As  one  example,  a  study  of 
engineers  (Slusher,  Van  Dyke,  &  Rose,  1972) 
found  that  die  most  highly  technically  qualified 
supervisors  were  least  likely  to  adopt  a 
managerial  role  and  in  turn,  were  lowest  in 
group  productivity  among  the  groups  studied. 

As  noted  in  die  introduction  of  this  technical 
report,  the  Federal  Aviation  Administration 
implemented  the  Air  Traffic  Supervisory 
Identification  and  Development  Program  (SIDP) 
to  change  the  emphasis  on  technical  performance 
in  promotabiiity  decisions  to  include  other  skills, 
such  as  decision-making  and  communication,  that 
reflect  supervisory  potential.  In  order  to  under¬ 
stand  ;*  'v  the  different  competencies  may 
distinguish  between  successful  and  unsuccessful 
candidates  at  different  phases  of  the  SIDP, 
comparisons  were  made  of  candidate  performance 
on  SIDP  performance  measures,  as  well  as 
measures  of  technical  performance  and 
experience.  The  influence  of  technical 
performance  and  skills  measured  in  the  SIDP  on 
success  in  each  of  the  phases  was  also  assessed. 
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METHOD 

Subjects 

Subjects  were  985  nonsupervisory  air  traffic 
control  specialists  who  entered  the  En  Route  or 
Terminal  Screen  Program  in  the  Academy  be¬ 
tween  1981  and  1985,  applied  to  SIDP  between 
1985  and  1988,  and  completed  their  field  training 
in  an  En  Route  or  Terminal  facility.  The  average 
age  of  this  group  at  the  time  they  applied  to 
SIDP  was  32.14  years  and  the  average  full 
performance  level  (FPL)  experience  was  3.69 
years.  Fifteen  percent  were  female  and  85%  were 
male.  All  applicants  were  required  to  have  at 
least  a  fully  successful  rating  on  their  last 
performance  appraisal;  29.6%  were  rated  fully 
successful,  52.5%  were  rated  exceptional,  and 
17.9%  were  rated  outstanding. 

Measures 

Peer-Supervisory  Assessment.  Applicants  to 
SIDP  identify  a  combination  of  peers  and  first- 
line  supervisors  to  complete  the  Peer-Super¬ 
visory  Assessment  (PSA).  A  paired  comparison 
approach  is  used  to  rate  applicants  against  other 
applicants  and  benchmarks  on  4  different 
dimensions:  Technical  Competence, 
Communication  Skills,  Interpersonal  Skills,  and 
Leadership.  Scores  are  computed  on  a  0  to  100 
scale;  the  composite  PSA  score  is  a  simple 
arithmetic  mean  of  the  4  dimension  scores. 
Successful  completion  of  this  hurdle  results  in 
referral  to  the  next  step  in  the  SIDP  process,  the 
Skill-Based  Interview  (SBI). 

Skill-Based  Interview.  The  SBI  is  a  combin¬ 
ation  of  a  face-to-face  interview  and  3  role- 
plays.  Three  interviewers  provide  consensus 
ratings  on  a  1  (weak)  to  5  (outstanding)  scale  on 
the  following  9  dimensions:  Organizational 
Knowledge,  Knowledge  of  Supervisory  Role 
Performance,  Organizational  Planning,  Problem- 
Solving  and  Analytical  Ability,  Judgment, 
Decision-Making,  Communication  Skills,  Inter¬ 
personal  Skills,  and  Leadership.  Based  upon  the 
combination  of  ratings  received  during  the  SBI, 
individuals  are  determined  to  be  (1)  "promotable" 
and  placed  on  the  Eligible  for  Consideration 
(EFC)  list,  allowing  them  to  apply  for  first-line 
supervisor  positions,  (2)  in  need  of  further 


skill  remediation  prior  to  placement  on  the  EFC, 
or  (3)  recommended  for  further  self-development 
and  reapplication  to  SIDP  at  a  later  date. 

Journeyman  Level  Technical  Performance. 
Technical  performance  at  the  journeyman  level 
was  identified  through  2  broad  measures:  (a)  the 
number  of  months  of  journeyman  level  ex¬ 
perience  the  applicant  had  at  the  time  of  applica¬ 
tion  to  SIDP  and  (b)  the  most  recent  performance 
appraisal  rating  prior  to  application  to  SIDP. 
Although  performance  ratings  can  actually  range 
from  1  (unacceptable)  to  5  (outstanding),  the 
program  requirement  for  a  fully  successful  (3) 
rating  to  apply  to  SIDP  restricted  the  range  to  the 
3  highest  possible  ratings. 

Two  different  sets  of  analyses  were  con¬ 
ducted  to  examine  3  different  groups:  (a)  whether 
the  individual  was  referred  to  the  SBI  or  not 
(Referral),  (b)  whether  or  not  the  applicant  was 
placed  on  the  Eligible  for  Consideration  list 
( Promotability ),  and  (c)  whether  or  not  the 
person  was  selected  for  a  first-line  supervisor 
position  ( Selection ).  Multivariate  analysis  of 
variance  (MANOVA)  was  used  primarily  as 
"protection"  from  an  inflated  alpha  level  on  the 
univariate  tests  comparing  PSA  and  SP!  com¬ 
posite  ratings  and  technical  performance  mea¬ 
sures  between  the  groups.  Although  statistically 
significant  differences  are  not  unexpected,  pri¬ 
marily  because  of  the  large  sample  size,  these 
differences  may  not  have  a  meaningful  impact 
on  the  likelihood  of  success  in  the  different 
phases  of  SIDP.  Thus,  logistic  regression 
analyses  were  conducted  to  determine  the  relative 
influence  of  the  SIDP  and  performance  variables 
on  the  probability  of  dichotomous  (i.e,  "pass- 
fail")  SIDP  outcomes. 

RESULTS 

Correlations  were  computed  between  each  of 
the  dimensions  rated  in  the  SIDP.  High  correla¬ 
tions  among  PSA  ratings  and  among  SBI  ratings 
(see  Table  1)  suggested  that  composite  variables 
for  the  PSA  and  SBI  might  be  needed.  Separate 
factor  analyses  (principle  components  and 
orthogonal  rotation)  were  conducted  for  the  PSA 
and  SBI  dimension  ratings.  Only  1  factor  was 
found  for  the  PSA;  scores  on  each  dimension 
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were  averaged  for  a  PSA  Composite  score.  Two 
factors  were  found  for  the  SBI  ratings:  a 
knowledge  factor  made  up  of  the  2  knowledge 
rating  dimensions  (Knowledge  of  Supervisory 
Role  Performance  and  Organizational 
Knowledge)  and  a  skill  factor  made  up  of  the 
remaining  seven  rating  dimensions.  Because 
ratings  cn  the  knowledge  dimensions  are  based 
on  answers  to  questions  rather  than  performance 
in  role-play  scenarios,  there  was  some  concern 
that  the  2  factors  reflected  method  variance 
rather  than  separate  dimensions.  Thus,  all  items 
were  included  in  an  item  analysis  using  the 
reliabilities  procedure  in  SPSS*  (Nye,  1990). 
Item  statistics  suggested  that  the  2  SBI  factors 
should  be  kept  separate  for  subsequent  statistical 
analyses.  Ratings  for  each  respective  factor  were 
summed  to  create  a  SBI  Knowledge  Composite 
and  a  Skill  Composite.  Correlations  between  the 
composite  variables  and  the  technical 
performance  variables  are  shown  in  Table  2; 
descriptive  statistics  for  the  variables  for  different 
group  comparisons  are  shown  in  Table  3. 

Referred  to  the  SBI 

Analysis  of  variance  (ANOVA)  was  used  for 
comparing  mean  differences  between  the  Referred 
and  Not  Referred  groups.  Only  the  Performance 
Rating  and  Months  FPL  Experience  were 
included  since  the  PSA  composite  operationally 
defines  who  will  be  referred  to  the  SBI  and 
would  be  expected  to  be  lower  for  the  Not 
Referred  group  than  the  Referred  group.  The 
Referred  and  Not  Referred  groups  had 
significantly  different  mean  Performance  Ratings 
(F=  101.81,  df~  1,983,  p<. 001)  but  were  not 
significantly  different  on  Months  FPL  Experience 
(F<  1.0). 

The  probability  of  being  referred  to  the  SBI 
was  estimated  using  the  SPSS*  logistic  regres¬ 
sion  procedure  (NoruSis,  1990).  In  the  first 
regression,  the  PSA  Composite,  Performance 
Ratings,  and  Months  FPL  Experience  were 
entered  into  the  regression  equation  using  the 
forward  stepwise  method.  The  second  regression 
excluded  the  PSA  Composite.  It  was  felt  that 
comparison  of  the  2  regressions  would  help  to 
identify  the  impact  of  the  peripheral  performance 
variables  on  the  probability  of  referral  to  the 


SBI.  Results  of  the  first  regression  showed  that 
only  the  PSA  Composite  and  Performance  Rating 
variables  predicted  the  referral  decision.  The 
goodness-of-fit  was  tested  using  -2  times  the  log 
likelihood  (-2  LL)  of  the  model  and  indicated  a 
good  fit  to  the  data  (-2  LL,  xJ(l, 982)= 555.42, 
p=  1).  The  overall  classification  rate  was 
89.85% .  When  the  PSA  Composite  was  excluded 
from  the  equation,  Performance  Rating  was  still 
identified  as  a  predictor.  However,  the  fit  to  the 
data  was  somewhat  poor  (-2  LL, 
x\ 1,983) =986. 10,  p=Al)  and  all  cases  were 
predicted  to  be  referred  to  the  SBI. 

Promotability 

A  multivariate  analysis  of  variance  test  was 
used  to  examine  mean  differences  on  the  PSA 
composite.  Performance  Rating,  and  Months  FPL 
Experience  between  those  who  were  placed  on 
the  EFC  list  (EFQ  and  those  who  were  not  (Not 
EFQ.  SBI  variables  were  excluded  from  this 
analysis  because  the  combination  of  rating  scores 
on  the  dimensions  operationally  determines  who 
is  placed  on  the  EFC  list.  The  multivariate  test 
was  not  significant. 

Again,  2  logistic  regression  analyses  were 
conducted.  The  first  regression  equation  in¬ 
cluded  the  SBI  Knowledge  and  Skill  Com¬ 
posites,  as  well  as  the  PSA  Composite,  Perfor¬ 
mance  Rating,  and  Months  FPL  Experience. 
Only  the  SBI  Knowledge  and  Skill  Composites 
predicted  the  promotability  decision.  The  good¬ 
ness-of-fit  was  tested  using  -2  times  the  log 
likelihood  (-2  LL)  of  the  model  and  indicated  a 
good  fit  to  the  data  (-2  LL,  *2(1, 742) =195.98, 
p- 1).  The  overall  classification  was  95.03%. 
Given  that  the  SBI  ratings  are  used  to  opera¬ 
tionally  determine  assignment  to  the  EFC  list, 
this  finding  is  not  surprising.  The  second 
regression  analysis  excluded  the  SBI  composite 
variables;  the  PSA  Composite  and  technical 
performance  variables  did  not  enter  into  the 
prediction  of  promotability  status. 

Selection 

The  dependent  variables  included  in  the 
MANOVA  were:  SBI  Knowledge  Composite, 
SBI  Skill  Composite,  PSA  Composite,  Perfor- 
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mance  Rating,  and  Months  FPL  Experience.  The 
MANOVA  test  was  statistically  significant  (Pi- 
llai’s  Trace  criterion,  F=4.00,  df=  5,543 
p<.001).  ANOVA  tests  demonstrated  signifi¬ 
cantly  different  group  means  on  the  PSA  Com¬ 
posite  (F=  13.56,  df=  1,547,  p<  .001),  the  SBI 
Knowledge  Composite  (F= 4.67,  df=  1,547, 
p<  .05),  and  Performance  Rating  (F=  4.74, 

1,547,  p<  .05). 

The  logistic  regression  analysis  included  all 
the  variables  used  in  the  MANOVA.  The  results 
showed  that  only  the  PSA  Composite  entered 
intothe  equation  for  predicting  selection  for  a 
supervisory  position.  Assessment  of  the  good- 
ness-of-fit  of  the  model  indicated  a  poor  fit  to  the 
data  (x2(l  ,547) = 605 .78,  p=.04).  Although  the 
overall  classification  rate  was  74.86%,  the  model 
predicted  that  no  one  had  been  selected  for  a 
supervisory  position. 

DISCUSSION 

The  results  of  the  analysis  of  the  referral 
variable  identified  a  significant  difference  bet¬ 
ween  the  Referred  and  Not  Referred  groups  on 
the  PSA  Performance  Ratings.  In  addition,  the 
PSA  composite  and  performance  rating  predicted 
referral  to  the  SBI.  Although  the  applicant’s 
performance  rating  is  not  considered  in  the  SIDP 
as  part  of  the  decision  to  refer  to  the  SBI,  it  may 
contain  information  that  is  redundant  with  peer 
and  supervisory  ratings  of  the  applicant  on  the 
PSA  dimensions.  In  fact,  performance  ratings  did 
show  small  but  significant  correlations  with  each 
of  the  PSA  dimension  ratings.  Thus,  the  perfor¬ 
mance  rating  may  reflect  similar  or  additional 
information  on  the  differences  among  applicants 
in  supervisory  and  technical  abilities. 

The  two  groups  considered  in  the  analysis  of 
promotabiiity  were  not  significantly  different  on 
the  PSA  Composite  or  technical  performance 
variables.  Of  the  independent  variables  used  in 
the  logistic  regression  analysis,  only  the  SBI 
composite  predicted  promotabiiity.  This  is 
perhaps  as  it  should  be,  since  the  SBI  is  meant  to 
identify  more  keenly  the  skills  rated  by 
supervisors  and  peers  in  the  PSA  (excluding 
technical  competence)  and  additional  important 
competencies  as  identified  in  a  study  of  FAA 


air  traffic  control  first-line  supervisor 
competencies  (Myers  &  Stutzman,  1991). 
Because  the  SBI  is  conducted  with  interviewers 
who  do  not  know  the  candidate  and  the  content 
of  the  scenarios  does  not  include  aspects  of  the 
air  traffic  controller’s  operational  job,  technical 
performance  (as  measured  by  performance 
ratings  and  experience)  should  not  play  a  role  in 
performance  at  this  stage.  Of  course,  an 
individual’s  performance  in  the  SBI  may  be 
enhanced  through  other  types  of  work 
experience,  for  example,  prior  staff  positions  or 
experience  as  an  on-the-job-training  or  Academy 
instructor.  These  types  of  experiences  bring  with 
them  opportunities  to  interact  with  others  to 
accomplish  tasks  unlike  those  normally  found  in 
the  air  traffic  controller’s  operational  position, 
and  can  enhance  current  supervisory  skills  and 
the  development  of  new  skills. 

Differences  between  those  who  were  selected 
for  a  supervisory  position  and  those  who  were 
not  were  found  for  the  PSA  Composite,  SBI 
Knowledge  Composite,  and  Performance  Rating 
variables.  Although  we  cannot  know  for  certain 
which  factors  the  selecting  official  is  emphasizing 
in  making  a  selection  decision  based  on  this 
analysis,  it  does  appear  that  several  factors  are 
being  considered  in  the  selection  of  supervisors. 
The  measures  for  which  there  were  significant 
mean  differences  encompassed  the  5  most 
frequently  mentioned  competencies  important  to 
task  accomplishment  (communication,  leadership, 
and  interpersonal  skills;  knowledge  of 
supervisory  role  performance,  and  technical 
competence)  identified  by  incumbent  air  traffic 
control  supervisors  (Myers  &  Stutzman,  1991). 
Thus,  selection  officials  appear  to  be  choosing 
individuals  for  supervisory  positions  in  line  with 
incumbent  perceptions  of  important  competencies 
in  performing  die  supervisory  job.  Given  the 
various  sources  of  the  ratings  (pews,  supervisors, 
and  trained  assessors)  selecting  officials  may  also 
be  considering  multiple  inputs  on  the  applicant’s 
qualifications  in  making  the  selection  decision. 
However,  these  results  must  be  considered  in 
light  of  the  fact  that  only  the  PSA  Composite 
entered  into  the  logistic  regression  equation  to 
predict  selection  status  and  the  model  did  not  fit 
the  data  well.  The  importance  of  different  factors 
in  attaining  a  supervisory  position  is  still  not  well 
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defined. 

Interviews  with  air  traffic  facility  selecting 
officials  conducted  during  an  evaluation  of  the 
SIDP  noted  that  factors  other  than  SIDP  perfor¬ 
mance,  such  as  diversity  in  work  experiences  and 
references  from  applicants’  supervisors,  play  a 
large  part  in  the  selection  decision  (Office  of  the 
Associate  Administrator  for  Human  Resource 
Management,  1991).  Identification  of  important 
factors  in  the  selection  decision  is  needed  as  part 
of  the  validation  of  the  SIDP.  In  addition,  collec¬ 
tion  of  information  on  applicant  experiences, 
education,  and  other  biodata  might  suggest  what 
additional  types  of  factors  may  relate  to  success¬ 
ful  performance  in  SIDP.  This  type  of  informa¬ 
tion  could  be  provided  to  applicants  to  improve 
their  chances  for  success  in  the  selection  pro¬ 
gram,  as  well  as  improving  their  effectiveness  as 
a  supervisor. 

Obviously,  this  study  did  not  span  the  do¬ 
main  of  possible  predictors  of  success  in  SIDP 
and  selection  as  a  first-line  supervisor.  Future 
research  in  areas  such  as  biodata,  achievement 
motivation,  and  personality  factors  is  needed  to 
contribute  to  the  theoretical  base  for  managerial 
potential  and  assist  in  improving  the  operation 
of  SIDP. 
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Table  1 

Correlations  among  technical  performance  measures,  PSA  ratings,  and  SBI  ratings 


Table  2 

Correlations  between  performance  variables 


1)  PSA  Composite 


2)  SBI  Knowledge 

.13' 

3)  SBI  Skill 

.07 

.44' 

4)  Performance  Rating 

.25* 

.00 

.03 

5)  Months  to  FPL 

-.04 

-.03 

-.01 

Measures 

1 

2 

3 

Table  3A 

Mean  scores  by  referral  status 


Referred  to  SBI 
(N  =  746) 

Not  Referred 
(N  =  239) 

Measures 

M 

SD 

M 

SD 

Performance  Rating 

4.00 

.67 

3.52 

.57 

Months  FPL  Experience 

44.46 

13.75 

43.56 

14.79 

Table  3B 

Mean  scores  by  promotability  status 


Eligible  for  Consideration 
(N  =  549) 

Not  Eligible 
(N  =  197) 

Measures 

M 

SD 

M 

SD 

PSA  Composite 

64.13 

10.80 

63.07 

11.59 

Performance  Rating 

Months  FPL  Experience 

4.01 

44.46 

.67 

13.75 

3.98 

44.52 

.67 

14.04 

Table  3C 

Mean  scores  by  selection  status 

Selected 
(N  =  138) 

Not  Selected 
(N  =  410) 

Measures 

M 

SD 

M 

SD 

PSA  Composite 

67.03 

11.38 

63.16 

10.44 

SBI  Knowledge  Composite 
SBI  Skill  Composite 

8.09 

30.61 

1.45 

2.90 

7.91 

30.33 

.67 

2.84 

Performance  Rating 

Months  FPL  Experience 

4.12 

45.20 

.70 

13.03 

3.97 

44.18 

.66 

13.83 

♦U.S.GPO: 1992-661-063/40076 
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